QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation
Authors: Yaoyu Zhu
Di Huang
Hanqi Lyu
Xiaoyun Zhang
Chongxiao Li
Wenxuan Shi
Yutong Wu
Jianan Mu
Jinghua Wang
Yang zhao
Pengwei Jin
Shuyao Cheng
shengwen Liang
Xishan Zhang
Rui Zhang
Zidong Du
Qi Guo
Xing Hu
Yunji Chen
Venue: NeurIPS 2025 poster ()
Keywords: LLM
Code Generation
Verilog
Primary area: deep_learning

Abstract: Large language models (LLMs) trained via reinforcement learning with verifiable reward (RLVR) have achieved breakthroughs on tasks with explicit, automatable verification, such as software programming and mathematical problems. Extending RLVR to electronic design automation (EDA), especially automatically generating hardware description languages (HDLs) like Verilog from natural-language (NL) specifications, however, poses three key challenges: the lack of automated and accurate verification environments, the scarcity of high-quality NL-code pairs, and the prohibitive computation cost of RLVR. To this end, we introduce CodeV-R1, an RLVR framework for training Verilog generation LLMs. First, we develop a rule-based testbench generator that performs robust equivalence checking against golden references. Second, we propose a round-trip data synthesis method that pairs open-source Verilog snippets with LLM-generated NL descriptions, verifies code–NL–code consistency via the generated testbench, and filters out inequivalent examples to yield a high-quality dataset. Third, we employ a two-stage "distill-then-RL" training pipeline: distillation for the cold start of reasoning abilities, followed by adaptive DAPO, our novel RLVR algorithm that can reduce training cost by adaptively adjusting sampling rate. The resulting model, CodeV-R1-7B, achieves 68.6 \% and 72.9 \% pass@1 on VerilogEval v2 and RTLLM v1.1, respectively, surpassing prior state-of-the-art by 12$\sim$20 \%, while even exceeding the performance of 671B DeepSeek-R1 on RTLLM. We have released our model, training code, and dataset to facilitate research in EDA and LLM communities.

QiMeng-CodeV-RI: Reasoning-Enhanced Verilog
Generation
Yaoyu Zhu', Di Huang 3 Hanqi Lyu'.2, Xiaoyun Zhang! CChongxioo Li
Wenxuan Shit.3, Yutong W Jianan Jinghua Wang', Yang Zhao
Pengwei Jin', Shuyao Cheng', Shengwen Liang', Xishan Zhangl.4 1.4
Rui Zhang', Zidong Du Qi Guo', Xing Hu', Yunji Chen
State Key Lab of Processors, Institute of Computing Technology, CAS
2 University of Science and Technology of China University of Chinese Academy of Sciences Cambricon Technologies
https://iprc-dip. gi github.ion io/CodeV-Ri.
Abstract
Large language models (LLMMs) trained via reinforcement learning with verifiable reward (RLVR) have achieved breakthroughs on tasks with explicit, automatable verification, such as software programming and mathematical problems. Extending RLVR to electronic design automation (EDA), especially automatically generating hardware description languages (HDLs) like Verilog from natural-language (NL) specifications, however, poses three key challenges: the lack of automated and accurate verification environments, the scareity of high-quality NL-code pairs, and the prohibitive computation cost of RLVR. To this end, introduce Code V-R1, an RLVR framework for training Verilog generation LLMs. First, we develop a rule-based testbench generator that performs robust cquivalence checking against golden references. Second, we propose a round-trip data synthesis method that pairs open-source Verilog snippets with LLM- -eMMeaeeetted NL descriptions, verifies code-NL-code consistency via the generated testbench, and filters out inequivalent examples yield a high-quality dataset. Third, employ a two-stage "distill- then-RL" training pipeline: distillation for the cold start of reasoning abilities, followed by adaptive DAPO, novel RLVR algorithm that can reduce training cost by adaptively adjusting sampling rate. The resulting model, Code 1- 7B, achieves 68.6 % and 72.9 % pass@) I on VerilogEval v2 and RTLLM v1.1, respectively, surpassing prior state-of -he-arr by 12~20 %. while even exceeding the performance of 671B DeepSeek-RI on RTLLM. We have released our model, training code, and dataset to facilitate research in EDA and LLM communities.
Introduction
Large language models (LLMs) have recently demonstrated remarkable progress on reasoning tasks when trained via reinforcement learning with verifiable reward (RLVR). Notable examples include OpenAI-ol [26] and DeepSeek-RI [3], which exhibit emergent reasoning capabilities on problems endowed with explicit verification procedures- such as software programming and mathematical problem solving. This success suggests a promising opportunity apply RLVR for electronic design
x Corresponding author. Contact: /zhuyaoyu, huangdi, huxing] @ict.ac.en. Please refer to https://iprc-de diw..iihuu.io//oooeee.. for relative resources.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

automation (EDA), specifically to the automatic generation of hardware description languages (HDLs) like Verilog from natural- -anguage (NL) specifications [38].
However, the three foundational components required for effective RLVR - (i) a reliable verification
environment, (ii) high-quality NL-code data, and (iii) an efficient training algorithm each present significant challenges in training reasoning LLMs for Verilog generation:
(1) Automated verification of hardware designs remains difficult. RLVR requires verification environment capable of providing accurate rewards. However, even in the data-rich software coding domain, such environments are rare. For example, most problems in the programming-come contest dataset APPS [6] have only one or two sets of unit tests, and they exhibit a false-positive rate of up to 60% when evaluated with an average of 20 unit tests [12]. Consequently, the software community has adopted the practice of using L.MS to generate additional unit tests in order to improve verification quality [11, 41]. Nevertheless, this approach is both costly and of limited effectiveness for hardware designs, because LLMSs lack the hardware-specifi. knowledge needed to handle the complex state spaces and corner cases of sequential circuits. For example, if the reset and clock signals are not correctly configured, the intended functionality cannot be properly verified.
(2) High-quality NL-code pairs for hardware designs are scarce. The proprietary nature of hardware designs severely limits the availability of annotated Verilog examples. Although several LLM-based methods have been proposed to synthesize NL-code pairs [5, 16, 45, 48], the resulting datasets often suffer from low-quality data (see Appendix D for examples), rendering them inadequate for RLVR's s stringent requirements.
(3) The computational cost of RLVR prohibitive. Training a 32B LLM on 1K data for 5 epochs using 16 NVIDIA H100 GPUs with supervised fine-tuning (SFT) takes only 0.5 hours [24]. In contrast, training a 14B LLM on 24K verifiable coding problems with reinforcement learning can take over 2.5 weeks on 32 NVIDIA H100 GPUs [20], making it prohibitively expensive to train Verilog reasoning LLM using RLVR.
To overcome these challenges, we introduce CodeV-R1, a comprehensive RLVR framework for Verilog generation. Our contributions are threefold:
(1) Automated testbench generation. We develop a rule-based testhench generation framework to verify the equivalence between a given Verilog implementation and its golden reference accurately as possible. For each golden reference, the framework first performs circuit-structure analysis to extract information such as input/output (I/O) ports and reset/clock signals. It then enumerates all reset and boock-yynchoonizttion scenarios to improve verification accuracy. Experiments demonstrate that our testbench achieves 96.1 % fewer false negatives than the LLM-generated counterpart and detects 62.5 % more injected errors in fuzzing tests for sequential circuits. Detailed experimental results are presented in Section 3.3.4.
(2) Round-trip data synthesis for high-qualit NL-code pairs. Leveraging our testbench generation framework, we propose the round-trip data synthesis approach that automatically synthesize high- quality NL-code pairs from code snippets. Specifically, candidate code snippets are first paired with LLM-generated NL descriptions, and then verified by regenerating the code from NL and comparing against the original for equivalence with our testbench. Only code that passes the testbench is retained and combined with the NL to form high-quality data for reinforcement learning. We theoretically prove that, given strong LLMS and an ideal verification environment, this procedure yields NL-code pairs of sufficiently high quality for RLVR with high probability.
(3) Two-stage training with adaptive DAPO for cost-effective RLVR. We adopt a two-stage "distill-then Siiill-then-LL- training pipeline to cold-start LLMs' reasoning ability through SFT and apply RL to enhance model's reasoning ability. Specifically, we use DeepSeek-R1 1 as the NL-tt-code L.M in our round-trip data synthesis produce (NL, Thought, Code) triplets, based on which perform SFT on our base LLM to obtain a distilled LLM with basic reasoning ability. Then, we apply RLVR on the distilled LLM using the seaiivalence-chekke high-quality data to further enhance its Verilog generation capability. Additionally, recognizing that RLVR's 's bottleneck lies in sampling and verification [20], we extend dynamic sampling policy optimization (DAPO) [44] with an adaptive mechanism that dynamically adjusts the number of samples per training step based on past sample discard rates. This approach notably reduces unnecessary sampling and verification overhead. thereby achieving 1.25x acceleration.
2

---

Natural Language Description
Reinforcement Learning (Adaptive DAPO)
Supervised Fine-Tuning
(Distillation)
Code-Verfiication
Thought
<ND sboul.
<NL> Toe matio
affiiiilyy Fibwring
<Testbench>
Round Trip Dato Synthesis
cley Ale cánswer>
officuity rheer
GodeV-811
S-Cinder
Eedel-at-Deell
Verilog Code
Verilog Code''
Earvolence Chocking by Testane)Tesllon))
Figure I: The overview of CodeV-RI. The core components of our framework include an automated testbench (Section 2.1), a supervised fine-tuning process (Section 2.2), and a reinforcement learning process (Section 2.3).
Based on these techniques, develop CodeV-RI-7B, specialized reasoning LLM for Verilog generation with only around 2,656 A 1000-GPU-hours. On the VerilogEval v2 [28] and RTLLM v1.1 v2 [19] benchmarks, Code V-R1-BB achieves Remarkably, it surpasses the BB DeepSeek-RI by 8.1 % on RTLLM v1.1 and 3.3% on RTLLM v2, demonstrating its strong RTL generation capabilities.
68.8 68.8% 72.9% / 68.0% pass @ respectively.
2 Methods
Our framework comprises 5 stages, including automated testbench generation framework (Fig- constitute the distillation phase, and stages and comprise the reinforcement learning phase. Below, we introduce the processes of these phases:
ure I). Stages
O Code-to-NL. Following prior work [47, 48], we collect Verilog code snippets from GitHub (de- noted y²) and use an LLM (DeepSeek- V3 [4]) to produce corresponding natural-lanuuage summaries (denoted z), creating an NL-code corpus ((x,, y with approximately 150K data samples.
NL-to-Code. Using DeepSeek-RI. we take each NL description x; from stage 1 and generate the "thought" (denoted c)) as well as Verilog code snippet (denoted y/), producing NL-thhughtt-ee thode code triples {(x,, 6,, }.
Supervised Fine-Tuning, We first filter the {(1,, 64 v() dataset by removing any examples for which base LLMS (e.g., Qwen2.5-Coder-77 Codde--B-IInstruee
1eeen.2.5-Cddrr3211 Instrrcct [9]) can generate correct code in any of 5 attempts (correctness verified using automatically generated testbench). We then perform SFT using these data on the base LLM to bootstrap their reasoning ability, yielding the distilled model, CodeV-R1-7DD -isttill. This stage uses approximately 87K examples.
Equivalence Checking. We use our automated testbench to verify equivalence between the original snippets y" and the newly generated snippets y' Any non-equivalent pairs {(2, v()) are discarded, while equivalent pairs are retained as high-quality data for subsequent RL training.
Reinforcement Learning, We again filter the retained ((ri, vt } set by removing any examples where the distilled model checked by the testbench). After this filtering, approximately 3. 3.1K examples remain. We then apply our adaptive DAPO algorithm, novel RLVR algorithm, to further improve Verilog--enerraion performance, resulting in the final model, CodeV-R1-7BB Next, we will describe in detail the automated testbench generation framework as well as the two training phases, distillation and RL.
generates correct code in all of 5 attempts (as
2.1 Automated Testbench Generation Framework for Verilog Code
To facilitate the rule-based reward mechanism for the RL process, we have developed specialized framework. This framework verifies the functionality of the generated Verilog code by conducting edge-triggere simulation and comparing it against the reference code. The verification framework unfolds in three consecutive phases:
Phase 1: Cireuit-Structure Analysis. Before performing functional verification, we extract the input/output (I/O) ports along with their respective bit-widths from the reference golden code using
3

---

Yosys [39]. For sequential circuits, we identify clock signals, noting their edge polarity (rising or falling), and characterize reset signals through control flow analysis. Reset signals are categorized based on synchrony (synchronous if they depend on the clock) and polarity (active-high or active-low).
Phase 2: Simulation. We simulate by providing random inputs to both the generated and the reference codes, and evaluating the equivalence of outputs. For combinational circuits, we employ M 100 independent simulation sequences for equivalence evaluation, each comprising N = 1000 inputs. Regarding sequential circuits, we adopt dual-stage validation approach when dealing with circuits that have either one reset signal or no reset signal at all: Firstly, we execute simulations using M = 100 sequences, each with N = 1000 clock toggles (500 cycles) with randomized inputs. In this stage, deterministic reset signals derived from golden reset behavior extracted via Yosys and representing expected, consistent reset logic- logic-are are applied at the start of each sequence, primarily aimed at testing the circuit's 's core funetionality. Secondly, we conduct simulations with an identical number of sequences and clock cycles with random reset signals, which validates the consistency of the reset signal operation. For circuit designs featuring multiple reset signals, we exhaustively test every non-conflicting combination, all maintaining the aforementioned MN cycle count.
2
Phase Verification. After each clock toggle, we assess the equivalence of the outputs between the generated Verilog code and the reference implementation. This process results in total of 2M N assessments for typical sequential circuits and MN assessments for combinational circuits. The Error Number x 100%. A value of = 0% indicates that the generated code functions correctly within our testbench environment. Through 32-way parallelization, the simulation achieves throughput of 15 instances per second.
verification outcome is quantified by an error rate metric e = 2MN
2.2
Code VodeV-R1-7B-Disillle Supervised Distillation for Verilog Data
Our pipeline for distillation begins with set of Verilog code (denoted as wt) collected from GitHub. We use DeepSeek-V V3 to summarize these code snippets, producing instructions xi corresponding to yt (stage Then, to produce the corpus for distillation, we ask DeepSeek-RI to generate responses containing "thought" cf and Verilog code snippet 3/ (stage These two stages yield approximately 150K NL-thought-code triples (I), d, v/))
Next, we curate a challenging subset through two filters: (1) retaining only instructions where Beeen2.5-Codee-7B---ne
fail to generate the code passing the functional verification (Section 2.1) to yf and (2) ensuring synthesizability of y with Yosys [39]. In addition, prevent benchmark contamination, we remove samples where the generated code yf exhibits Rouge-L similarity > 0.5 [13] to VerilogEval v1 [14] / v2 [28] or RTLLM v1.1 [19]/ v2 [17], yielding 87K high-quality samples (stage
baseline models
Finally, we initialize CodeV-RI-7B-Dise Distill from Bwwen2.-CCoder-7B- InIstruct and fine-tune it to generate complete responses (c/, v/) given I,, Following DeepSeek-RII Is methodology [3], maximize the likelihood of the generated responses using our prompt template (see Appendix E), with implementation specifies detailed in Section 3.1 (stage
2.3
V-ReV-: Reinforcement Learning on the Distilled Model
To further improve the model's reasoning ability, we perform reinforcement learning fine-tuning based on CodeV-R V-R1-7B-Disstt with carefully selected high-quality Verilog data (stage and stage 9). Below we will introduce our data curation method (Section 2.3.1), RL training algorithm, and reward design for RL (Section 2.3.2).
2.3.1 High-quality Data Curation
Experiences from prior research suggest that conducting RL training on problems that the model can solve but requires reasoning to address more effectively enhance the model's RL capabilities [34]. Furthermore, given potential inconsistencies between the golden code {ur] } in the original dataset collected from GitHub and the instructions {æ,} generated by DeepSeek-V3, we prioritize ensuring the correctness of selected problems. To summarize, our RL (question, answer) pairs must meet three key criteria: being solvable, challenging, and error-free.
To implement this framework, we identify problems where DeepSeek-R1 successfully generates code matching the golden one in the original dataset, while both Owen2.5-Coder: 7B-Instruct and Qwen2.5-

---

Coder-32B 3nn--nstuttt fail to produce equivalent solutions. Specifically, we conduct equivalence checking between the [ code generated by DeepSeek-Ri in the 87K dataset and {3? } in the original dataset, retaining only validated {(14, yi pairs for RI training..
For difficulty enhancement,
to generate five code variants per question, excluding cases where all generated codes match the golden one, as these reflect patterns already mastered during supervised fine-tuning (stage 1) Through this rigorous selection process, we curate a final dataset of 3. I. high-quality examples for reinforcement learning.
employ CodeV-R1-7B-Diise
Additionally, formalize the equivalence between code and natural language, and theoretically prove the effectiveness of our data curation. Intuitively, the Code-to-NL and NL-to-Code conversion process using LLMs inevitably leads some information loss. Therefore, if the converted code remains equivalent to the original one after back-and-fore conversion, the probability of error during the conversion process minimal. Detailed definition and proof are shown below.
Definition 2.1 (NL-Code Deterministic Equivalence (NLCDE)). Let F denote the space of all code snippets, L the space of natural-languag (NL) descriptions, and R. C F x L a semanticffunctional equivalence relation where (f.t)= R iff code f fully implements NL l (or precisely describes f)-
F (NL-to-code), the NLCDE states: For all f E F,, L: 1. If M1 generates with Pr(1 [) = 1, then (f,() E R (deterministic NL summaries are semantically equivalent to input code). 2. M2 generates f with Pr(S (t) 1, then (J,1) E R (deterministic code oulputs are functionally equivalent to input NL).
Consider two probabilistic models, M1 : F + L (code-to-NL) and M2 -
Theorem 2.1 (Semantic Equivalence in Round-Trip Transformations. Consider the probabilistic models M1 F - L (code-to-NL) and M2: - F (NL-to-code) from the NL-Code Deterministic Equivalence (NLCDE) definition (Definition 2.1). Let Y F be random code snippet drawn from some distribution, and define the transformed objects: X = M1 (Y) E C. Y' = M2(X) ) F. For any pair of objects A, B, let EAB denote the event "A and B are semantically equivalent." the round-trip transformation preserves equivalence with certainty under NLCDE, i.e., Pr Pr[ Ey,x' 1. then both forward and backward transformations are individually equivalent with certainty: Pr [ryx A Ex,y'] =
Proof Sketch. This theorem can be proved by the Data Processing Inequality. Please refer to Ap- pendix A.1 for the detailed proof and further explanation.
2.3.2 Adaptive DAPO Algorithm
We enhance the DAPO algorithm [44] with two efficiency improvements for RL fine-tuning on the distilled model (stage
The core DAPO loss operates on groups of G responses per prompt:
(D) = )x aa (-1z)
G
kil
1
min (Tir(0)An,, clip (r),, soow, 1 + Ehigh) An.)
Σ
|
(1)
s.t. Doglis__quivalentt yi, y "²) < <G,
where (0) ao(yelle.eee)) (1..1... Age- (a-mman((8 (R, is the reward to be introduced later) RR.(
Ti, the importance sampling ratio under the new policy me compared to the old policy meae As, is the relutvee advantage, |y.| is the length response to calculate token-level loss, and flow < Ehigh are asymmetric clipping thresholds introduced in DAPO to encourage exploration. The constraint 0 < iealissequivalenie yi, y H < G ensures each training batch contains both correct and incorrect responses. Note that we do not include the overlong filtering proposed by DAPO here.
key feature of DAPO the dynamic sampling mechanism. It notably improves the training result. However, the standard DAPO sampling strategy presents inefficiencies during sample generation: DAPO's fixed generation batch size (denoted as bgen) is suboptimal. If too few partially correct. samples are generated for the RL train batch size (denoted dtrain), costly re-sampling occurs; if
5

---

too many are generated, excess samples are wasted. This problem intensifies as training progresses, and the improvement in model accuracy reduces the number of partially correct examples.
We address this with an adaptive batch size mechanism utilizing a dynamically estimated sampling effective ratio, Toalid- Initially, bgen is set to birain- After successfully accumulating full training batch (btrain), we calculate the batch effective ratio number valid samples), The value of Teatid is then Ogen
updated to the minimum of itself and the batch effective ratio. For the subsequent sampling phase, the generation batch size is adaptively set to byen = herasn" 1. The detailed process is given in Appendix A.2. Note that this acceleration does not involve offline updates or alter the composition of the RL training batch, so it preserves DAPO's accuracy while accelerating training.
We implement a rule-based reward function that evaluates both structural correctness and semantic equivalence. A response y/s receives reward of 1 if it satisfies two conditions: (1) Proper formatting as wtt.r/awnnnnggnnnbb-emeewwwoooio
(2) Semantic equivalence with the golden code y" judged by the equivalence checker introduced in Section 2.1. The reward function R(y, v²) if y has a correct format and (y, yy" are functional equivalent, and otherwise.
3 Experiments
This section details the implementation of our method and presents comprehensive experimental results. We systematically evaluate our model through multiple dimensions: comparisons with prior state-of-the-art approaches, test-time scaling analysis across varying response length constraints, ablation studies analyzing the impact of golden code correctness and problem complexity, acceleration effects of the adaptive DAPO mechanism, and testbench performance evaluation. These analyses collectively demonstrate the effectiveness and efficiency of our proposed approach.
3.1 Implementation details
We obtain our final model by first distilling DeepSeek-RI and then applying RL on our curated 3.1K dataset. During distillation, we employ LLaMAFactory [50] to supervised fine-tune (SFT) Swen2.5-Coder--77 Instruct using the 87K dataset filtered for distillation. We train the model for epochs with a learning rate of 10-5 and a batch size of 64. The total context length set to 16384 during distillation. During RL, we use the verl [32] framework to further train the distilled model with our adaptive DAPO. We use batch size of 128, learning rate of 10 and train for 300 steps. The rollout temperature is set to 1.0. During this stage, the max length is set to 2048 for instruction and 16384 for response. The SFT stage executed on 8 A A100-80G GPUs, taking approximately 78 hours, while the RI. stage runs on 16 A A000 GPUs, requiring around 127 hours of computation. The whole parameter setting is provided in Appendix B.
We test our distillation and RL model on various Verilog benchmarks, including VerilogEval v1 [14] / v2 [28] and RTLLM v1.1 [19] v2 [17]. For VerilogEval v2, we examine zero-shot scenarios in both specification-Uu to-oonttt-----e RTL translation and code completion tasks. The maximum context length is configured to 16384 tokens during the evaluation phase for all benchmarks. The temperature during generation is 0.6 for the distillation model and 1.0 for the RL model, and 20 responses are generated per query to estimate the pass@1 @k score for both Evai and RTLLM.
3.2 Main Results
Our main experimental results shown in Table and Table 2. We evaluate DeepSeek-RL [3], DeepSeek-V33 V3 [4], QWQ-32B [36], SeepSeee--R--istillll 7B [3], W.ven25-Cooer-32BBB1
[3], DeepSeek-RI-Diin -i-rss-----ssttll---een [42], and GPT-4o [25] on
[42], --oel:55-odrr------ma VerilogEval and RTLLM. Meanwhile, we adopt results reported by RTICCder [16], BetterV [45],
CodeV [48] CrafRTTL [15] from their papers. The results demonstrate that:
Our model achieves state-of-the-art (SOTA) performance among Verilog-domain models on most benchmarks. Our model has a significant advantage over previous Verilog-domain models on RTLLM v1.1, outperforming the previous SOTA model, CraftRTL--DS66.77 by 18.8% on the pass @ I metric. On VerilogEval v1-Human, although the performance improvement compared to the previous SOTA model, CraftRTL-SC2-157 is not substantial, our model has smaller size (7B) compared

---

Table I: Comparison of CodeV-B1-7B against baselines VerilogEval v1 and RTLLM v1.1.
Type
Model
Open SOUrEO
Verilggvvl--Mate (5) pass
VerilogEval-Humen (%)
RTLLM (5)
puss
pass
pass
passii
puss11
pass
GPT-4a"
67.7
75.5
77.2
61111
71.4
74.5
41.7
65.9
DeepSeek-11-6718
81.0
87.4
59.5
$1.5
88.5
64.8
82.9
DorpSask-V3-66188
87.5
68.7
79.7
K2.1
609
Foundation
QWQ-328*
71.1
84.0
87.0
63.8
78.0
81.3
50.9
70.6
Models
DeepSeck-RR -R----Diiillieetttt
64.7
80.5
83.6
51.3
68.1
72.2
42.1
64.3
DeepSock-
Qwen-7B-
16.9
66.6
76.6
79.7
47.6
58.
61.8
17.9
67.7
Owen2.5 Swee2.5-Coom S-Cddrr------ Instruce
60.2
77.8
82.4
31.9
46.3
50.2
32.2
48.2
72.3
36.7
RTI.Code--DS-6.76
61.2
76.5
81.8
41.6
50.1
53.4
48.3
BetterV-CL-78
64.2
75.4
79.1
40.9
50.0
53.3
Reteevv-DS-6.78
Specialized
BelcrV-OQQ-B
68.1
79.1
84.5
46.1
53.7
58.2
Models
CodeV-DS-6.7B
CodeViCL-7B
V
77.9
78.1
88.6
86.0
90.7
S8.5
45,2
52.7
62.5
59.5
63.8
67.3
39.4
12.4
62.1
55.2
CodeV.CQ-7B
98.2
x
78.
K5.5
87.8
63.1
67.8
69.7
42.6
$2.9
CruRKTI--D8-4.7 8B
77.8
85.1
88.1
65.4
70.0
72.1
53.1
58.8
Ours
Oxdev-g1-7--Diiill
Cod-V-R1-7B
Y
76.5
76.2
K5.6
84.1
85.7
87.0
65.7
69.9
79.3
76.8
79.7
81.7
72.9
57.4
75.8
86.1
We evaluate the models with *, while other results are sourced from their papers.
Table Comparison of CodeV-RI-7B on VerilogEval v2 and RTLLM v2.
Model
Open
paseli
NisilogEvllZSSR (S)
pass
10
pas-11
Veriong EvalD-CCCCCCCCC (S)
pars@)
XILLM v2 (%)
passfi
GPT-e
Dex/6ecl-R1-671B
85.1
79.7
71.3
Foundaiion
QWZ-10R
77.8
DeepSeek Qwen
DrepSeek Qwen
Owee2.5 Coder
0wee2.5
Instruct
30.5
46.5
57.6
Specialined Medels TTTLCCdee-DS66.77
31.1
47.8
52.3
33.7
49.8
49.2
Oura
CodeV.R1-7B-Disit CodeV-R1-78 2B
65.2
08.8
78.2
75.2
81.1
77.5
65.5
69.9
75.6
78.2
75.2
80.9
57.2
68.0
78.2
71.9
$1.7
77.1
We evaluate all models in this table. SR: Specification-to-TTI RR-L CC: Code Completion.
to theirs (15B). Among 7B models, we outperform the previous best model (CraftRTTL--D-6675) by 4.5% on pass@1 1. Although our model does not perform well on VerilogExal- this benchmark relatively easy, and even DeepSeek-RI does not have significant advantage on it.
Our model demonstrates superior performance over most foundation models across both benchmarks. Although does not surpass the DeepSeck-RI model-the the primary source for knowledge distillation most benchmarks, it consistently exceeds the performance of other foundation models. key finding that after applying reinforcement learning (RL), our model outperforms DeepSeek-RI
both RTL.M--1.. and RTLLM-v2, underscoring the significant efficacy of the RL. phase. The inderwhelming results of other foundation models, such as Qwen2.5- Coder-Instruct and DeepSeek-RI B1tteekk-R--iistiie 1-Dis-iill- -iee, highlight the limited exposure to Verilog data during their pre-training and iustruction-tuning stages. This is further evidenced by the observation that distilling general-purpose knowledge from large models (e.g., in mathematics and software code) fails to enhance the Verilog capabilities of smaller models.
Reinforcement learning significantly improves model performance. Compared with Code 7B-Distill, our RL model CodeV-R1-77 shows a noticeable improvement on almost all benchmarks. Especially on the RTLLM benchmark, the reinforcement learning process results an improvement of over 10 % for the pass@I score. This indicates great potential of RL for Verilog code generation and showcases the robustness of our testbench in providing reliable functional correctness rewards.
3.3
Additional Experiments
3.3.1
Test-Time Scaling
Test-time scaling is an important ability of reasoning L.MSs [24]. To verify the test-time scaling ability of our CodeV-RI V-R1-7B,, we take the RTLLM v1.1 dataset as an example and evaluate the accuracy of model and DeepSeek-RR RI under varying response length budgets. Formally, we force the response
7

---

Testfime Scaling ATLLM v1.1
Test-Time Scaling RTLLM v1.1 (PLOPS)
Oer Despleeck)
DeepSnnk 41
(14)
6.04
36.0%
10.4%
0.1
g
10%
0.
10%
104 19th
18
10"
Response Length Budget
FLOPs
(a)
(b)
Figure 2: Test-time scaling RTLLM v1.1. Figure (a) shows response length against accuracy, while Figure (b) shows FLOPs against accuracy. FLOPs are estimated according to model architecture.
length of both models to be smaller than certain thresholds (4096, 8192 and 16384 tokens), and plot the corresponding results in Figure 2a. To ensure fair comparison, we also normalized FLOPs consumption at each response length, shown in Figure 2b.
Both models' accuracy improves considerably as the response length budget increases from 4096 to 16384. CodeV-R1-7B3 accuracy rises from 7.1% % to 72.9%, outperforming DeepSeek-R] (29.0% + 64.1%)). When evaluated in terms of FLOPs efficiency, CodeV-RI-7B demonstrated superior computational economy, delivering higher accuracy per unit of computation compared to DeepSeek- RI. These results underscore Code V-R1-7B exceptional test-time scaling efficiency, showcasing its ability to leverage longer contexts more effectively than DeepSeek-Rl R1 while consuming fewer computational resources on the RTLLM v1.1 benchmark.
3.3.2 Equivalence Checking and Difficulty Filtering Improves RL Training
To explore whether equivalence checking and difficulty filtering improve RL dataset quality, we conduct an ablation study by constructing two additional datasets.
Our original RL dataset contains 3.JK problems where DeepSeek-R1 responses pass the equivalence checking, while both C-nne..-Cddrr-B-- sampling attempts. To conduct difficulty ablation, we introduce dataset without difficulty filtering containing 16K problems, where we additionally include samples where Qwen2.5 models succeed in some attempts under our testbench. To conduct reference code correctness ablation, we introduce a dataset without round-trip equivalence checking containing 14K samples, where we treat DeepSeek-RI outputs as pseudo-golden code. We select cases where Qwen2.5 models fail to match this pseudo-golden code in five attempts control difficulty. To avoid time waste, we filter the Distill has 100% pass rate under our testbench in five attempts.
and Swven.55-Code--e Inntruct fail across five
problems where
turo3
rcundierpp eqgatie sancE cheeddra dificuity filoring (196)
Responce Length
10:00
0.43
dificcitty
Training Steps
Training Steps
(b)
Figure 3: Train-time scale up some key metrics. Figure (a) tracks response length, whereas Figure (b) presents the corresponding trend for reward.
We perform reinforcement learning using Maaee-R1-B---iitee on the three aforementioned datasets, employing identical training parameters. Key metries observed during these training processes are presented in Figure 3. Inspection of Figure 3a reveals distinct trends in response length during training. Utilizing the original RI. dataset leads noticeable subsequent increase response length, whereas the training dataset without difficulty filtering leads to segment of response decrease.

---

This suggests that even when initial responses are relatively long, incorporating more challenging samples during reinforcement learning facilitates further steady growth in response length. Figure 3b illustrates that the pseudo-goldes -ooddegg dataset consistently exhibits notably lower reward throughout the training process compared to our original RI. dataset. This underscores the critical role of golden code accuracy during reinforcement learning.
3.3.3 Acceleration via Adaptive DAPO
To quantitatively demonstrate the acceleration achieved by our adaptive DAPO algorithm, provide a comparison of time usage in Figure 4a. The plots reveal a notable increase in the time per RL step in baseline DAPO training around step 150. This performance degradation in the baseline is attributed to its fixed generation batch size, which becomes insufficient to yield enough samples for complete training batch as training progresses. In contrast, our adaptive DAPO effectively mitigates this issue. It dynamically adjusts and increases the generation batch size across steps. In addition, when generation attempt does not produce sufficient valid samples for training batch, the algorithm recalculates the required remaining batch size. In Figure 4b, we provide the average speedup of adaptive DAPO, along with a breakdown performance before and after step 150. Notably, the time reduction after step 150 is significantly more pronounced- the speedup factor reaches 1 1.44 after step 150, compared to .04 before step 150. This disparity highlights the critical benefit of eliminating sampling more than once. After applying adaptive DAPO, the final speedup factor reaches 1.25x.
Time Comparison Between
DAPO Baseline
Stop Time Comparison Between Adaptive DAPO and Easeline
Adastie DATO Seuciin DAPO
2390.44
Adaptinc De-O
Base DAPO
AGU 1850
221
1500
Belore Steps
After Sleps
Mad (300 Steps!
(a)
(b)
Figure 4: Time comparison between adaptive DAPO and baseline DAPO. (a): Comparison of RL training time per step. (b): Acceleration ratio between adaptive DAPO and baseline DAPO, breakdown by step (whether before 150).
3.3.4 Testbench Performance Evaluation
We evaluate our auto-testbench generation framework against DeepSeek- V3-generated testbench, both taking the Verilog code from GitHub the golden reference. We conduct two key tests:
Correctness classification test. We assess whether the testbenches might misclassify correct code as incorrect. To do this, we validate with "golden vs. golden" inputs (i.e., comparing the golden code against itself). The expected outcome is 100% correct classification. Our method misclassifies only 0.3% of cases (due to problems of Icarus Verilog simulation and randomization issues) 96. 1 % reduction false negatives compared to the DeepSeek-V3-gennraaee V3-generated testbench (7.6%).
Fuzzing test for sequential circuits. Second, we perform fuzzing test on sequential circuits by instructing DeepSeek-V3 to inject subtle errors into the golden code. The goal is to measure how effectively each testbench detects these mistakes. Our testbench detects 65% of the injected errors, demonstrating a 62.5% relative improvement in detection rate over the DeepSeek V3 testbench (40%) and indicating fewer false positives.
Related Work
4.1 Large Language Models for Reasoning
The OpenAT-ol [26] series the first closed-source model trained with large-scale reinforcement learning to perform reasoning through COT. Inspired by its powerful and effective reinforcement

---

learning training paradigm, QwQ [36], DeepSeek-RI [3], and Kimi k1.5 [34] have all adopted and improved upon its approach, achieving promising results. Limited by computational resources, open-source communities have actively explored low-cost approaches to replicate ol ol-like reasoning models. Some efforts have focused on distilling the powerful closed-source reasoning models [8, 23, 24, 35, 43]. while others have also explored training reasoning models using reinforcement learning [7, 10, 18, 20, 21, 27, 40, 46].
The main difference between CodeV-RI-7B and the aforementioned reasoning models lies in its focus on hardware description language code generation, which poses unique challenges due to verification difficulty and limited data quality. In contrast, prior works primarily specialised in domains such as mathematics, which benefit from easily verifiable numerical outputs and rich open-source datasets.
4.2 Large Language Models for Verilog Code Generation
With the development of large language models, specialised code generation models for hardware description languages also receive widespread attention. Many prior works [2, 15, 16, 45, 48] focus on Verilog instruction-tuning data creation without a striet correctness evaluation. Most works have a syntax check in constructing instruction-response pairs: RTLCoder [16] and Code [48] add syntax checks when constructing supervised fine-tuning (SFT) datasets with closed-source LLMs. BetterV [45] maps code across languages using Verilog syntax constraints, while OriGen [2] leverages compiler feedback to eliminate syntax errors. For functional correctness, to date. only CraftRTL's correct-by srrrectby-coostructfe
approach [15] ensures functional correspondence between instruction and response through formal verification. However, its applicability remains restricted to Karnaugh maps and finite-state machines, a narrow subset of Verilog design challenges.
This verification bottleneck shifts to the model optimization stage. Specifically, reinforcement learning with rule-based rewards attempts address functional correctness by relying on testbenches for reward calculation. However, this strategy is undermined by the fact that current testbench generation paradigms suffer from two systemic flaws: (1) Unverified validation frameworks: For example, VeriPrefer [37] optimizes testbench coverage, but its testbenches themselves may be flawed, sometimes failing to pass the reference code they were designed to verify. Reasoning V [29] co-generates code and testbenches via DeepSeek-R1. inheriting the model's hallucination risks. (2) Cost-prohibitive iteration: AutoBench [30] and CorrectBench [31] employ multi-stage LLM workflows, where each self- ceff-correction cycle incurs escalating computational costs and latency, directly conflicting with RL's demand for rapid, low-cost reward feedback.
Unlike prior work, we apply Verilog functional verification with auto-generated equivalence checking (see Section 2.1), providing robust foundation for both data curation and reinforcement learning.
5 Conclusion
In this paper, we propose Code V-R1, a unified RLVR framework designed for training Verilog gener- ation LLMs. This framework first distills data with reasoning patterns and then applies reinforcement learning on high-quality data curated by automated testbench generation framework. The model trained via this framework, Code V-Re 1-7B, achieves outstanding performance on Verilog generation benchmarks like VerilogEval and RTLLM, matching or even surpassing DeepSeek-RI. which demon- strates the effectiveness of the automated testbench generation and the two-stage training paradigm. Notably, to the best of our knowledge, CodeV-R1-77 is the first Verilog-domain LLM to demonstrate strong reasoning capability showing consistent test-time scaling behavior. A series of analytical experiments further highlights the powerful impact of Code framework in enhancing data quality and further unlocking the Verilog code generation capabilities of LLMs through reasoning.
10

---

Acknowledgements
This work is partially supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (Grants No.XDB0660300, XDB0660301, XDB0660302), the NSF of China (Grants No.62341411, 62222214, 62525203, U22A2028, 6240073476), CAS Project for Young Scientists in Basic Research (YSBR-029) and Youth Innovation Promotion Association CAS.
References
[1] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint vrXiv:2107.03374, 2021.
[2] Fan Cui, Chenyang Yin, Kexing Zhou, Youwei Xiao, Guangyu Sun. Qiang Xu, Qipeng Guo, Yun Liang, Xingcheng Zhang, Demin Song, et al. Origen: Enhancing rtl code generation with code-to-code augmentation and self-reflection. Proceedings of the 43rd IEEF/ACM International Conference on Computer-Aided Design, pages 1-9, 2024.
[3] DeepSeek-Al. AI. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025. URL https: //tp:///aaiivavea wtt.oor//////oo//ooooo abs//501. 12948.
[4] DeepSeek-AL Deepseek-v3 technical report, 2025. URL httt:/ss//aaxxi..rrgaab///////// 19437.
[5] Mingzhe Gao, Jieru Zhao, Zhe Lin, Wenchao Ding, Xiaofeng Hou, Yu Feng. Chao Li, and Minyi Guo. Autovcoder: A systematic framework for automated verilog code generation using Ilms. In 2024 IEEE 42nd International Conference on Computer Design (ICCD), pages 162-169. IEEE, 2024.
[6] Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. Measuring coding challenge competence with apps. arXiv preprint
2021.
[7] Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung- Shum. Open-reasoner- An open source approach scaling up reinforcement learning on the base model, 2025. URL
org/ abs/2503 24290.
[8] Zhen Huang, Haoyang Zou, Xuefeng Li, Yixiu Liu, Yuxiang Zheng, Ethan Chern, Shijie Xia, Yiwei Qin, Weizhe Yuan, and Pengfei Liu. 01 replication journey - paat Surpassing op-preveew through simple distillation, big progress or bitter lesson?, 2024. URL https: //arxiv. org/ abs 22bb/ 16489.
[9] Binyuan Hui, Jian Yang. Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. Qwen2.5-coder technical report, 2024. URL https: //arxiv. org/ abs/ /209 12186.
[10] Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training Ilms to reason and leverage search engines with reinforcement learning. arXiv preprint 2rX 16, 2025.
[11) Rongao Li, Jie Fu, Bo-Wen Zhang, Tao Huang, Zhihong Sun, Chen Lyu, Guang Liu, Zhi Jin, and Ge Li. Taco: Topics algorithmic code generation dataset. arXiv preprint arXiv:2312. 14852, 2023.
[12] Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d' Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang. Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. Competition-level code generation with alphacode. Science, 1796622) 2022. doi: 10. 126/science. abql158. URL https: / //www/www.//// science. www.eenee.rr//daaeeeeeee httg//oi/abbs.. 1126/ science. abg.

---

[13] Chin- Yew Lin. Rouge: A package for automatic evaluation of summaries, In Anmual Meeting of the Association for Computational Linguistics. 2004. URL https:///ppp http://pp..saaaiicccccciies semanticscholas org/ org/CorpusID 9642877
[14] Mingjie Liu, Nathaniel Pinckney, Brucek Khailany, and Haoxing Ren. Verilogeval: Evaluating large language models for verilog code generation. In 2023 IEEF/ACM International Conference on Computer Aided Design (ICCAD), pages 1-8. IEEE, 2023.
[15] Mingjie Liu, Yun-Da Tsai, Wenfei Zhou, and Haoxing Ren. Craftrtl: High-quality synthetic data generation for verilog code models with correct-by-construction non-textual representations and targeted code repair. arXiv preprint arXiv:2409. 22933, 2024.
[16] Shang Liu, Wenji Fang, Yao Lu, Jing Wang, Qijun Zhang, Hongce Zhang, and Zhiyao Xie. Rtlcoder: Fully open-source and efficient Ilm-assisted rtl code generation technique. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2024.
[17] Shang Liu, Yao Lu, Wenji Fang, Mengming Li, and Zhiyao Xie. Openllm-rtl: Open dataset and benchmark for Ilm-aided design rt generation. In Proceedings of the 43rd IEEE/ACM International Conference Computer-Aided Design, pages 1-9, 2024.
[18] Zhaowei Liu, Xin Guo, Fangqi Lou, Lingfeng Zeng, Jinyi Niu, Zixuan Wang, Jiajie Xu, Weige Cai, Ziwei Yang, Xueqian Zhao, Chao Li, Sheng Xu, Dezhi Chen, Yun Chen, Zuo Bai, and Liwen Zhang. Fin-rl: A large language model for financial reasoning through reinforcement 16252.
learning, 2025. URL
org/
[19] Yao Lu, Shang Liu, Qijun Zhang, and Zhiyao Xie. Rull:: An open-source benchmark for design rtl generation with large language model. In 2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC), pages 722-727. IEEE, 2024.
[20] Michael Luo, Sijun Tan, Roy Huang, Ameen Patel, Alpay Ariyak, Qingyang Wu, Xiaoxiang Shi, Rachel Xin, Colin Cai, Maurice Weber, Ce Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepcoder: fully open-source 14b coder o3-mini level. https: http://tettttrrrtaeee b75. notion. site/ Rilte/DDeee /DeepCoder A- A-Fully- -pen---------- Source ce- 14B--oodr--- at -0- mini-Level.
2025.
Notion Blog.
[21] Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa. and lon Stoica. Deepscaler: Surpassing ol-preview with 1.5b model by scaling rl. https:/ http/prettyrrrada rrdi--adii-b- b75. notion. site/ SeepScaleR-Surpassiiie Serraassinng---Peeee -wwe---eee-------eee-e------ a 5B Modeedel------------------ee--e- -by Soy-Scaling ing-Rn, RL, 2025. Notion Blog.
[22] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, (NNov):5579-2605 2008
[23] Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng. Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems, 2024. URL httttpss//aaxiv.rrgggg /24/2 09413.
[24] Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501.19333, 2025.
[25] OpenAl. Gpt-4o system card, 2024. URL
21276.
[26] OpenAl. Openai ol system card, 2024.
[27] Jiayi Pan, Junjie Zhang, Xingyao Wang, Lifan Yuan, Hao Peng, and Alane Suhr. Tinyzero. https//gthhhhcommmajaaiaa////aip 2025. Accessed: 2025-01-24.
[28] Nathaniel Pinckney, Christopher Batten, Mingjie Liu, Haoxing Ren, and Brucek Khailany. Revisiting verilogeval: Newer 1lms, in-context learning, and specification-to tasks, 2024. URL https: aariiiiiiaaaaaaa//aaaaaaaaaaa..a org/ abs///bs/2440 2000b. 11053.
12

---

[29] Haiyan Qin, Zhiwei Xie, Jingjing Li, Liangchen Li, Xiaotong Feng, Junzhan Liu, and Wang Kang. Reasoningy: Efficient verilog code generation with adaptive hybrid reasoning model, 2025. URL https:tp::.. htapps://arxiie /ttpx/xxvv.o////// perrgabs//2504. 14560.
[30] Ruidi Qiu, Grace Li Zhang. Rolf Drechsler, Ulf Schlichtmann, and Bing Li. Autobench: Automatic testbench generation and evaluation using Ilss for hdl design. In Proceedings of the 2024 ACM/IEEE International Symposium on Machine Learning for CAD, pages 1-10, 2024.
[31] Ruidi Qiu, Grace Li Zhang. Rolf Drechsler, Ulf Schlichtmann, and Bing Li. Correctbench: Automatic testbench generation with functional self-correction using llms for hdl design, 2024.
URL htttp:://aattii org/ abss/2111 08510.
[32] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rihf framework. arXiv preprint arXiv: 2409.1999999 19256, 2024.
[33] Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah Smith, Luke Zettlemoyer, and Tao Yu, One embedder, any task: cntructtio-finetunee text embeddings. arXiv preprint arXiv:2212.09741 2022.
[34] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.
2025.
[35] NovaSky Team. Sky-tl; Train your own ol preview model within S450. https:/novvssy- n.gichub.io/posst/kkkykll 2025. Accessed: 2025-01-09.
[36] Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL github. itriii//llll http//lll/////////////////////gl 32b/
[37] Ning Wang, Bingkun Yao, Jie Zhou, Yuchen Hu, Xi Wang, Nan Guan, and Zhe Jiang. Insights from verification: Training a verilog generation Ilm with reinforcement learning with testbench feedback, 2025. URL https: // arxiv org/ /2o./ 15804.
[38] Ning Wang, Bingkun Yao, Jie Zhou, Xi Wang, Zhe Jiang, and Nan Guan. Large language model for verilog generation with code-structurr-guidee reinforcement learning, 2025. URL nttps://ares /ttp://aaxvvv.// wwtesiiv.r////ooo org/ ab2/ 18271.
[39] Clifford Wolf, Johann Glaser, and Johannes Kepler. Yosys-a free verilog synthesis suite. In fli 2013. URL https: / //api. semanticscholar wacplaarrrrr//oooee parg/CCrruusDD... 202611483.
[40] Tian Xie, Zitian Gao, Qingnan Ren. Haoming Luo. Yuqian Hong. Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing Ilm reasoning with rule-based reinforcement learning, 2025. URL httph:/ /awp/v//a/vvvv/aa//////eaaeaiava org/ /2022/225..... 14768.
[41] Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, and Radha Poovendran. Kodcode: di- verse. challenging, and verifiable synthetic dataset for coding. arXiv preprint wxXh:2503..02511, 2025.
[42] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412.155 15115, 2024.
[43] Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint arXiv:2502.03387, 2025.
[44] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint ar arXiv:2503. 14476, 2025.
13

---

[45] PEI Zehua, Huiling Zhen, Mingxuan Yuan, Yu Huang, and Bei Yu. Betterv: Controlled verilog generation with discriminative guidance. In Forty-first International Conference Machine Learning, 2024.
[46] Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models the wild, 2025. URL https: / arxiv org/ wwwa/asss/// 2503. 18892.
[47] Yongan Zhang, Zhongzhi Yu, Yonggan Fu, Cheng Wan, and Yingyan (Celine) Lin. MG- Verilog multi-grained dataset towards enhanced llm-assisted verilog generation. In The First IEEE International Workshop on LLM-Aided Design (LAD 24), 2024.
[48] Yang Zhao, Di Huang, Chongxiao Li, Pengwei Jin, Ziyuan Nan, Tianyun Ma, Lei Qi, Yansong Pan, Zhenxing Zhang, Rui Zhang. Xishan Zhang, Zidong Du, Qi Guo, Xing Hu, and Yunji Chen. Codev: Empowering Ilms for verilog generation through multi-level summarization,
2024. URL https: /atll//araa aarxiv..vvv. org/ aaaaaass/244 10424.
[49] Yujie Zhao, Hejia Zhang, Hanxian Huang, Zhongming Yu, and Jishen Zhao. Mage: A multi- agent engine for automated rtl code generation. In 2025 62nd ACM/IEEE Design Automation Conference (DAC), pages 1-7. IEEE, 2025.
[50] Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. Llamafactory: Unified efficient finettuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) pages 400-410, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction accurately describe the paper's contributions, including the equivalence checking, dataset filtering based this equívalence checking, and adaptive DAPO. These are detailed in Section (Introduction) and Section (Methods).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected generalize other settings.
It fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The limitations of this work primarily include the cost of data synthesis and potential issues with low-quality data, which are discussed in the Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Liimitttion" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: Theorem 2.1 (Semantic Equivalence Chain Transformations) is provided under the NLCDE assumption (Definition 2.1).
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
The paper describes the dataset creation process, RL algorithm, evaluation metrics, LLMs used, experimental setup including SFT, RL, and evaluation parameters (Sections 2 and 3).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of model checkpoint, other means that appropriate the research performed.
also be provided via detailed instructions
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (c.g., a large language model), then there should either be a way to access this model for reproducing the results or way reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers to have some path reproducing verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We have released our complete model, code, and dataset on Huggingface (model/dataset) and Github (code).
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. cc/ for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed reproduce the results. See the NeurIPS code and data submission guidelines (https: //ipps.ccpputeeee mublic/guides/
waweeesummssiooooiiiccooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized ver- sions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 3.1 (Experiment Implementation Details) has introduced key parame- ters of SFT, RL, and testing. Besides, the full parameter setting provided in Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The paper reports performance metrics as percentages and raw counts in tables such as Table However, it does not include error bars, confidence intervals, or formal statistical significance tests for these results. Conducting multiple runs for each experimental condition to gather data for robust statistical significance analysis was deemed prohibitively expensive. This is primarily due the high cost of RL training.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the emror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of eπrоrs is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
This paper claims computer resources for SFT and RL in section 3.1.
Guidelines:
The answer NA means that paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: httpp://neett /ttpe///nnuupaeeeeeee cttp:ppubbcc/chhcccciieeeeeeeeed
Answer: [Yes]
Justification: The research focuses on LLM--ased Verilog code generation, using publicly available code from GitHub create training dataset. It does not involve human subjects, sensitive personal data, or applications with immediate high-risk ethical concerns outlined in the NeurIPS Code of Ethics. The goal is advance Verilog auto-generation.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
18

---

Justification:
See Introduction and Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considcrations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses addition to attacks, mecha- nisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The training data used in this work comes from open-source code GitHub and the open-source language model (e.g., DeepSeek-RI), with no risk of sensitive content leakage.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper properly credits the creators of the datasets used: VerilogEval v1 [14] / v2 [28] and RTLLM vI.1 [19] v2 [17] are cited in Section 3. Also the creators of the models used: DeepSeek-R1 [3], DeepSeek-V3 [4] and Qwen [9] are cited in Sections 1 and 2. According to public sources, all datasets and models are released under open-source licenses, and their use in this paper complies with the respective licensing terms.
19

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (c.g., website), the copyright and terms service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged reach out the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces the CodeV-R1 training framework, and the model trained by it, CodeV-R1-7B in Section The paper itself serves as the primary documentation for these new assets. If these assets were to be released, further practical documentation (e.g.. README files) would typically accompany them.
Guidelines:
The answer NA means that the paper does not release new assets. Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The research described in the paper does not involve crowdsourcing experi- ments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
20

---

/LLM) for Answer: [NA]
Justification: The research does not involve human subjects and therefore did not require IRB approval an equivalent review.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: The use of Large Language Models (LLMs) such DeepSeek-RI, DeepSeek- V3, and Owen2.5-Coder: 7B is a fundamental and core component of this research. The entire paper is about "reasoning LLMs for RTL generation" defined as using LLMs to generate RTL code, and proposes an "training framework" specifically to train a better LLM in this task. This detailed throughout the paper, particularly in Sections (Introduction), 2 (Methods), 3 (Experiments).
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (tttts://neuttii http://eeeisccc//eee cctooeeerrecceeeeccccceceeessese http://///esaeeeeeeo//o//a/e//// what should or should not be described.
21

---

A Method Details
A.1 Proof and Further Explanation of Theorem 2.1
Proof. Observe that the sequence Y + X - Y' forms a Markov chain. By the Data Processing Inequality (DPI),
1();Y) S X();)).
Under the assumption that Eyy,, holds almost surely, we have H(Y Y") = 0, and thus
1(Y;Y)) H(Y) - HY Y") = H(Y).
follows that
H(Y) = I(Y;Y) & X(: & H(Y) = I(Y;)) = H(Y) = H(Y x) = 0,
meaning Y is determined by X almost surely and hence Ey,x holds.
Next, since H(Y | X) implies (X) = H(Y) and I(X;Y') S H(X), a failure of Ex,y
under the NLCDE assumption would force
I(X;Y') < (X) HY)).
contradicting I(Y;Y') -YY). Therefore, Ex,y" must also hold almost surely. Combining these two results gives
Ey,x A Ex,y' holds almost surely.
The need for NLCDE in Theorem 2.1 arises because Ex,y = H(Y X)= = = 0, but when and Y are incorrectly matched with probability one. To be more specific (though not fully rigorous, just to aid understanding), if the NL-to-code model wrongly transforms A (e.g., "design a multiplier") in the NL domain to B (e.g.. "design an adder") in the code domain, and transforms B in the NL domain to A in the code domain, while the code-to-NL model maps A in the code domain to the NL domain and B the code domain to A in the NL domain, then (Y YY) 0 can hold without y Thus, the NLCDE assumption is necessary to resolve this.
Remark:
H(Y X) = A Ex,y. A counterexample
Further explanation: Here we re-emphasize some critical points of this theorem:
1. Functional Identity:
The theory built upon the space of code functions (F) and natural language descriptions (C). Different code snippets / NL descriptions that implement / describe the same function (e.g., the same RTL module) are considered identical within F/C. L.
2. Interpretation of Determinism: The assumption of deterministic mappings (M1 : F + L and M2 L F) models the high functional consistency (not textual uniformity) achieved by capable LLMs. As model capability increases, the mapping from precise NL description a core code function becomes increasingly stable (probability converges to 1).
3. Theoretical Relevance: Theorem 2.1 establishes that, under these idealized conditions, the functional equivalence (of Y and Y') after the round-trip process (Y + X - Y') can guarantee the correctness of both problem summarization (Y + X) and code generation
(X + Y'). This provides a foundational principle explaining why our synthesis loop can bootstrap high-quality, self-consistent data, principle that is strongly supported by our empirical outcomes.
A.2 Algorithm Description of Adaptive DAPO
In this section, we provide the algorithm description of adaptive DAPO in Algorithm 1. In this algorithm, one epoch means going through the whole training dataset, while one step is collect enough samples and update the model parameters like standard DAPO [44]. Note that we achieve the dynamic batch size by two granularities: First, we use a step-level ratio Toalid to control the generation batch size byen- Second, if one generation does not provide enough samples for training, we use another inner-step-level ratio sseep control the generation batch size for the remaining samples.
22

---

Algorithm Adaptive DAPO
Require: Training batch size btrain. dataset !
Ensure: Updated Tvalid and filtered problem pool
Initialize "valid f-
for epoch 1, 2, do
Shuffle D
(Epoch reset)
Ntotal 5- (D), Nonnsumed f- 0
while Neonsumed Neotal do
(Process epock)
Ebgen + O, Ryalid &- I step +- Tealid
while Avalid < Utrain do
Dremain &- braain Thyalid
bge +- Civemminn/sste
(Dynamic batck)
D' + D-D[Nconumees
min Neonsumed + bge, NNota))
Generate bye samples from D
Update counters: Tyalid f- Rvalid + Unew. Dbgen + Mben + bge
Tstep f --min (Tstep: Dbyen
end while
Update ratio: rvalid +- min (ealldd Edgen
Auniist
Train DAPO with btrain valid samples end while end for
(RL step)
B Parameter Setting
The full parameter setting during the SFT (distillation) stage shown in Table 3, while the full parameter setting during the RL stage is shown in Table 4. During testing, use max context length of 16384 and a temperature of 1.0. We set top_p to 1.0 for VerilogEval and 0.95 for RTLLM.
For RL, the generation batch size in Table 4 corresponds to train batch_bathh____ batch_size size in verl [32]. and the training batch size corresponds to ppo_mmp mini batch size in verl. generation batch size of 128 and training batch size of 64 (with a rollout number of 16) means first generating 128 16 samples for 128 problems and updating two times, each with 64 x samples, during one RL step. Meanwhile, the clip ratio,,igh), clip ratio(low), overlong penalty factor, and overlong response length in Table are introduced by DAPO. Here, the max train response length Table corresponds to Immaz in DAPO, and the overlong response length corresponds to Leache. The overlong penalty in DAPO Piength (y) (where is response length) is defined as:
0,
|y| S Limax Leache Lmax Leache ly S Lmax Lmna < 11),
Piength (1)
[y] (Immas Teachee) Loache
(2)
which is added to the (0, 1} reward.
Table 3: SFT Parameter Setting.
Parameter Category
Parameter Name
Value
Parameter Name
Value
Training Mode
Finetuning Type
Full Paramcter
Deepspeed
Zero3
Epochs
6
Learning Rate (LR)
10
Optimization & Scheduling
Batch Size
64
Optimizer
AdamW
LR Scheduler
Cusine Decay
LR Warmup Ratio
0.03
Numerical Precision
BF16
Context & Data Handling
Max Context Length
16384
Packing
True
23

---

Table 4: RL Parameter Setting.
Parameter Category
Parameter Name
Value
Parameler Name
Value
Generation Batch Size
128
Training Batch Size
Batch Size Related
Dynamic Batch Size
True
Rollout Configuration
Kollout Number
16
Rollout Temperature
Rollout Engine
VLLM
Rollout GPL Memory Utlization
0.8
Optimization Regularization
Learning Rate
TX 10
Weight Decay
0.0
KL Coctficient
0.0
KL Loss Coefficient
Ratio (High)
0.28
Clip Ratio (Low)
Clipping Penalty
Overlong Penalty Factor
1.0
Train Response Length (Fall)
Overlong Response Length
Length Control
Max Generate Response Length
32768
Computation & Memory Optimization
Gridient Clip
Use Liger Kerne
0.5
True
VLLM Enforee Eager
Gradient Checkpointing
False
True
Tensor Parallel Size
Distributed Training Confgurationn Data Processing FSDP Related
Number Nodes
GPUs per Node
Remove Padding
Tree
Token Level Loss
Tree
FSDP Optimazer Ofllodd
False
FSDP Parameter Offload
False
C Additional Statistics and Analysis
C.1 Benchmark Comparison
Since there is notable performance gain difference (especially for the RL phase) of method between VerilogEval and RTLLM,, we provide a deeper analysis of this phenomenon this section. Given that RTLLM's performance gains stem mainly from reinforcement learning, we focus distribution differences between our RL dataset and the two benchmarks.
Distribution similarity to RTLLM: We run the instructor--emeedding model [33] for the golden code in our RL dataset, RTLLM (v2), and VerilogEval (v2 spec-to-RTL), then generate t-SNE distribution plot [22] in Figure 5. This plot revealed that our RL dataset aligns closely with RTLLM but diverges from VerilogEval for both problems and solutions.
T-SNE Distribution of Problems
T-SNE Distribution of Solutions
Data Source
Data Source
Cacwloi1
veriogEval
Dimencion 2
Dimendion
Dimension 1
Dimension
(a)
(b)
Figure 5: T-SNE distribution CodeV-RI RL dataset, RTLLM (v2), and VerilogEval (v2 spec-to-
RTL). Left: Problem (NL) distribution; Right: Solution (code) distribution.
Table 5: Centroid Distance and Similarity between Our RL Dataset and Benchmarks
RTLLM v2 VerilogEval v2
Euclidean Distance
0.1
0.2903
Cosine Similarity
0.9926
0.9547
We also show cosine similarity and Euclidean distance metrics for the embedding centroids between our RL. dataset and the benchmarks in Table Our RI. dataset's embedding centroid shows signifi- cantly smaller Euclidean distance and closer-to-1 (maximum value) cosine similarity with RTLLM

---

than with VerilogEval. Importantly this reflects only embedding centroid relationships, not data homogenization or overfitting to RTLLM.
Problem type difference: and identified that one reason for this performance gain difference is VerilogEval's heavier use
Additionally, we conducted detailed case study across both benchmarks
of cable/graph-basee problems, where our model underperforms significantly. Table 6 presents comparison of our model's accuracy against DeepSeek-RI on these problem types, including their ratios within benchmarks:
Table 6: Accuracy Comparison on Table/Graph Problems Across Benchmarks
Trbpes/Graphs (Ours) Tables/Graphs (DS-RI)
VerilogEval v2 (Ratio)
29.49%
29. 29.49%
EvailogEval v2 (Accuracy)
55.43%
78.04%
RTLLM v2 (Ratio)
0%
0%
RTLLM v2 (Accuracy)
N/A
N/A
This points to key improvement direction: incorporating more table/graph--peccfii instruction- response pairs (c.g., KMap, FSM, waveform data as in CraftRTL [15]) into our training dataset.
Benchmark complexity comparison: Additionally, we provide the token count comparison. serving as proxy for complexity. for these two benchmarks. Table 7 provides the average number of lines and tokens (comments and blank lines removed) for VerilogEval and RTLLM. From these results, we can clearly see that RTLLM is generally more complex than VerilogEval.
Table 7: Code Length Comparison between VerilogEval and RTLLM Datasets
Dataset
Average Lines
Average Tokens
VerilogEval v1 Machine
13.89
93.85
VerilogEval v1 Human
15.82
117.35
VerilogEval v2 Code Completion
16.10
121.84
VerilogEval v2 Spec to RTT.
16.12
122.31
RTL.MM v1.1
56.52
470.48
RTLLMM v2.0
46.30
403.80
C.2 Additional Benchmark Statistics
In this section, we take a close look the mistake type VerilogEval v2 and the pass @k metrics of different task types on RTLLM v2.
Table 8: Comparison of Error Types for VerilogEval v2.
Compiler Errors
Runtime Errors
Model
C
w
Total
R
T
Total
CodeV-RI-7B-Disilll
69
107
108
20
0
332
1699
107
19
CodeV-RI-7B
46
1
1610
114
1
1725
DeepSeek-R1-671B
37
47
144
1096
5
1211
Error type explanation: C- General Compiler Error; S Syntax Error; Reg Declared as Wire;
Module Missing; p Unable to Bind Wire/Reg; e Explicit Cast Required; SSensitivity Problem; c
Unable to Bind Wire/Rog 'clk"; ; R General Runtime Error; T T Timcout. r- Reset Issue;
Mistake type: As shown in Table 8, our RL training notably reduces error rates, particularly for compiler errors. CodeV-RR-7B achieves a 48% reduction in total compiler errors compared to CodeV-R1-7B-Diist (from 332 to 173), with the most pronounced improvements in syntax errors (S, reduced by 65% from 107 to 38) and wire declaration issues (w, down 57% from 108 to 46). Notably, our CodeV-R1-78 has a remarkably fewer syntax error (38) compared to DeepSeek-RI (59) and fewer reset issues (r) (1 vs 5). Even so, our CodeV-R1-77 1-7B still has limitations. For instance, the number of general runtime errors (R) is still notably higher than DeepSeek-RI. This might stem from the RL training data not being suitable for VerilogEval (unlike the great improvement on RTLLM).
25

---

Table 9: Performance Across Different Module Categories RTLLM v2.
Arithmetic (9k)
Control (S)
Memory (%k)
Miscellaneous (S)
Model
pass
pass 6
pass 1
pass
pass
pass
pass@1
pass@5
CodeV-R1-7B-Diiile
69.47
89.19
74.17
83,06
43,57
56.69
46.94
59.72
CodeV-R1-7B
83.68
91.66
80.00
83.33
51.43
63,30
57.78
72.40
DeepScek-RI R1-6718
76.58
90.65
83.33
83.33
57.14
60.71
52.50
67.72
Accuracy among task types: Table 9 demonstrates the comparative performance across module categories, where CodeV-R1-78 shows consistent improvements over CodeV-RI-7B-DD while maintaining competitive results against the larger DeepSeek-R1. Notably, Code 1-7B achieves superior pass rates in all categories over CodeV-R1-7B-Distiil, with particularly strong gains in arithmetic modules (83.68% vs 69.47%) and miscellaneous modules (57.78% vs 46.94%) It also surpasses DeepSeek-RI in these two categories. Compared with the training dataset classification provided in Figure these two categories occupy a larger portion (arithmetic and others). This observation suggests that augmenting the training set with high-quality RL data for currently under- performing categories (particularly Memory and Control modules) could be productive direction for future model improvement.
C.3 Training Dataset Statistics
(a)
(b)
Figure Problem category distribution. Left: SFT dataset; Right: RL dataset.
Figure 6 presents the category distribution of our 87K SFT and 3.1K RI. training datasets (categorized using both questions and answers). While both datasets show comparable distributions, the RI. dataset has fewer unclassified problems.
Prompt Length Distribution Analysis
SFT Data Prompt Length Distributton
RL Data Prompt Length Distribution
00000
7/00
2/50-
25000
2350
20000
750
10000
5000
250
Figure 7: Prompt length distribution. Left: SFT dataset; Right: RL dataset.
Figure 7 illustrates the prompt length distribution (in tokens) for our 87K SFT and 3.1K RL training datasets, both clipped to a maximum prompt length of 1500 tokens. The figure reveals a sharper distribution for the RL data, indicating shorter and lower-variance prompt lengths compared to the
26

---

SFT data. To quantify this observation, we calculated the following statistics: The average length of SFT data is 377.81 with a standard deviation of 161.30, while the average length of RL data is 336.67 with standard deviation of 153.88. These statistics align with the visual trends in the figure.
Model Response Length Distribution Analysis
6000
2000
10.00
1263
1900
2003
300M
1200
2000
1000
4000
1867
12800
1000
1067
Figure 8: Response length distribution. Left: SFT dataset; Middle: Incorrect samples in RL. dataset.
Correct samples in RL dataset;
Right:
Figure depicts the response length distributions (in tokens) for CodeV-R1-7B-Dise and Code R1-7B. Note that the maximum context length-the the sum of prompt length and response length- is capped at 16384 tokens. Consequently, when responses are truncated, their recorded length 16384 tokens minus the prompt length, resulting in somewhat scattered distribution (manifested as the two rightmost bars, instead of one, become longer in the distribution plot). The response length for CodeV-R1-7B exhibits evident right shift, indicating longer responses after reinforcement learning. Additionally, Code 1-B's ' response distribution is more symmetric compared to the left-skewed distribution of investigation. We observe that incorrect samples are significantly longer, with a substantial proportion exceeding the length threshold. Even excluding these overlong samples, incorrect responses remain longer, characterized by a higher peak value (Code V-R1-7B) or a slower post-peak decline (Code R1-7B-Distill). An intriguing phenomenon is that Code has lower overlong ratio on the RL dataset but a higher ratio on the SFT dataset. This may arise from overfitting the overlong penalty during RL, while CodeV-RI-7B'ss 's tendency to generate longer responses increases overlong instances on the SFT dataset.
The underlying cause of this discrepancy warrants further
C.4 Agent Ability Analysis
This section presents supplementary experiments on agentic integration capabilities for our Code R1-7B with the MAGE [49] framework. We identify two fundamental distinctions between MAGE and our approach: (1) MAGE utilizes golden testbenches for verification while CodeV-RI operates without them, and (2) MAGE's performance exhibits strong dependency on both prompt engineering strategies and the underlying model capabilities.
Hence, we employ a targeted experimental design to isolate and evaluate CodeV-R1's 's efficacy within multi-agent system. Specifically, replaced only the RTL generation agent in MAGE with CodeV-RI R as this role best aligns with CodeV-RI' ''s core capabilities while maintaining the other agents based on
The experimental results in Table 10 demonstrate that CodI achieves competitive performance (94.87% Pass @1 1) compared to the berf-perg model (Claude-3.5-Some at 95.51%), while sub-
27

---

stantially outperforming other baseline models. This indicates CodeV-RI's potential for integration into agent workflows and suggests promising directions for developing end-to-end agent capabilities in future work.
Table 10: Agent Performance Comparison in MAGE Framework
Model
Pass
Deepseek-R1-0525
80.77%
DeepSeek-V3
88.46%
GPT-40-240806
76.28%
Sruude-3.5-Sonne
95.51%
CodeV-RI Claude)
94.87%
D
Case Study
Comparison Between CodeV-R1-7B and DeepSeek-R1: In this section, we first present a case study on specific Verilog problem in VerilogEval-v2 to illustrate the advantage of CodeV-RI-7B over DeepSeek-RI this problem. The problem, reasoning chains, and results are shown in the blocks below.
Both Deepseek-F RI and our model comprehended the problem and engaged in reasoning and reflection, but only our model accurately implemented the logical function defined by the Karnaugh map, while R1's answer executed an incorrect logical function. The key distinction lies in our model's use of the declaration impul [4:/] x, which directly corresponds to the notation of the Karnaugh map the problem description. In contrast, RI employed inpul [3:0] x, leading to confusion in subsequent reasoning. Although our model was initially challenged by this unconventional declaration, ultimately chose the correct declaration through reflection, avoiding potential index mapping ambiguities. This resulted in significantly clearer code that is less prone to errors.
Problem Description
Question:
Question: would like you implement module named TopModule with the following interface. All input and output ports input x (4 bits) output
bit unless otherwise specified.
The module should implement the function f shown in the Karnaugh map below. d is don't t-care, which means you may choose to output whatever value is convenient.
x[1]x[2]
x[3]x[4]
00
d
d
01
101
11
11111d11
[111101d]
Answer:
28

---

Reasoning Chain (DeepSeek-RI)
Okay, so need to write a Verilog module called TopModule that takes a 4-bit input and outputs a single bit I' based on the given Karnaugh map. Let me start by understanding the problem step by step.
Wait, K-map's rows are x3x4, columns xlx2. Let's see. For example. the first row is x3x4 row is determined by bits and x4. Similarly, columns are determined by x and x2. So cach cell in the K-map is determined by the combination of x3, x4, x1, x2. So the four input bits are x1, x2, x3, x4. But how are these mapped to the input in Verilog? The input x is 4-bit vector, Let's assume that is declared [3:0] x. where x[3] the first bit (MSB) and x[O] the last. So perhaps the bits ranged x[3] = x1, x[2] x2, x[1] = x3, x[O] x4. But this a guess. Alternatively, the problem might have typo, and the rows are [[2]x[3], and columns x[O]x[1]. But given the problem statement, need to procoed with the given labels.
Now, the goal cover the and O's with the largest possible groups, using don't c-rers either or to maximize grouping-
First, let's identify the I's and O's. Looking for I's:
In row 01 (x3=0, x2=1), column 11 (xl=1, x0=1) +
row (x3=1, x2=1), columns and 01 - I's.
In row 10 (X3=1, x2=0), columns 00, 01 - I's.
In row 10, column 11 + O. Wait, no: row 10, column 11 (x1=1, x0=1) is 0. So that's 0. The 0's are:
Row column 01 0.
Row 01, columns 00, +
Row 10, column 11 - 0.
The don't '-cares can be used to simplify the expression. So for f, can choose to treat the don't "t-cars as 1 or to form the largest possible groups.
Therefore, the Verilog code for the module would be: module TopModule input [3:0] x, output f);
assign f = (x[3] & -x(1)) [-x[3] & x[1] & x(0));
endmodule
But let me double-check this. Another way check is test some values. Test case 1: x3=1, x2=1, x1=0, x0== - should be 1.
In the K-map: row x3x2=11 (row 11), column x x1x0=00 1. Our expression: (1 & 0 0)
10 = 1. Correct.
Test case 2: x3=1. x2=0, x1=0, x0=1 row 10. column - Expression: (1 & 1) (0 & Conect.
Test case 6: x3=0, x2=0, x0=1, row 00, column 11 (don (don't-care). Our expression sets this I via x3 x0. So f=1 here. Which acceptable a don't-care.
Thus, Verilog code should correctly implement the K-map.
Results (DeepSeek-RI)
**Solution**
"verllog
module TopModule (
input [3:0] x,
output f
):
assign (x[3] & -x(I]) (-x[3] x[1] x(0]);
endmodule
29

---

Reasoning Chain (CodeV-RR-7) V-R1-7B)
<think>Okay, let's try to tackle this Verilog problem. The goal to create a module called TopModule with a 4-bit input and single-bit output f. The function is determined by the Karnaugh map provided.
me try to find the logic.
Looking for the minterms where f=1, considering don't cares as Is.
Wait the f=1 in the K-map.
Looking across the K-map, the entries that have are: In x3x4=11
Columns xlx2 when:
x x1x2=00 (f=1) and x1x2=01 (f=1).
In x3x4=10 row:
xlx2=00 and 01 (so x3x4=10, xlx2=00 and 01 which gives xl=0, x2=0 and xl=0, x2=1. So x3x4
can be and xlx2-00 and 01.
Also:
Row x3x4=01 and x1x2=11: f=1.
Row x3x4=01's 's column x1x2=11 (third column) gives
The variables are x[I]. x[2]. x[3]. x[4]. Wait input is four bits. Let's suppose the four bits are x[3:0]. but the code might have x[3]. x[2). x[I], x(0]. but the naming here may vary. Wait the K-map labels the as x[1] to x[4]. So in Verilog, input is declared as [4:1], which a stan- dard way.
But in Verilog, arrays usually declared with [width-1
but the actual variable name is [high:low]. example, input [4:1] x would have indices x[4), x[3], x[2], x[1]. x[O].
But in the Karnaugh map, the rows are x3 and x4, columns are x1 and x2. But perhaps in the Verilog code, the variables are named x(1 x[I]
Let's assume the input is
four-bit vector named x. with each bit labeled x1, x2, x3, x4, such that: input [4:1] x; x[1J. x(2), x[3], x[4] But in Verilog, you can declare it as: input [1:4] x; // hut the syntax is different.
Altemnativly., input [4:1] where x[1] is the first bit, etc.
But given the problem statement's variables x[1] to x[4], the code he written module TopModule (input [4:1] x, output D): Then, the variables are: x[I]. x[2]. x[3], x[4].
So the final code would look like: module TopModule (input [4:1] x, output
assign [x]3] & -x[1]) (-x[3] x x[4] x [1] x [[]));
endmodule Wait wait, the variables in the second term are x3=0 (because x[3], x4=1 (x[4)), xl=1 (x[I).
x2=1 (x(2)).
Yes.
So that's correct.
</think>
Results --RI-BB
<answers verilog
module TopModule (input [4:1] x, output D;
assign == (x[3] & -x(I]) I (-x[3] & x[4] & x[I] & x(21);
endmodule
</answer>
Low-quality Data Example: Below is example of low-quality data in our dataset. The original
code is just an empty module with some comments, while the summarized problem describes module with an unsigned 32-bit divider. The problem and the code are inconsistent in this case.
30

---

Original Code from GitHub
module div unsigned acir.
clock
denom.
numer.
quatient..
remain);
imput aclr:
input clock;
input [31:0]
denom;
input [31:0]
numer:
output [31:0]
guotient;
output [31:0]
endmodule
// Retrieval info: PRIVATE: new diagram STRING "1"
11 Retrieval info: LIBRARY: 1pm 1pm 1pm caam___oooonene. all
DREPRESENTATINE STRING "UNSIGNED" 11 Retrieval info: CONSTANT: IPM HINT STRING "MAXIMIZE, SPEED=6, REMAINDERPOSTTIV-TTRRF
Retrieval info: CONSTANT:
11 Retrieval infe: CONSTANT: LPM VREPRRSSENATIIN STRING "UNSIGNED"
11 Retrieval info: CONSTANT:
Retrieval info: CONSTANT: LPM PIPELINE NUMERIC "I1"
TYPE STRING
DIVIDE"
11 Retrieval info: CONSTANT: LPM WIDTHD NUMERIC "32"
11 Retrieval info: CONSTANT: IPM WIDTHN NUMERIC "32"
11 Retrieval info: USED PORT:
acir
0 INPUT NODEEVAL "acir"
H Retrieval info: USED PORT:
clock
INPUT NODEFVAL "clock"
11 Retrieval info: USED PORT:
denom O 32 INPUT NODEFVAL "denom [31.. 01"
HI Retrieval info: USED PORT:
numer O O 32 INPUT NODEFVAL "numer[31... 01
Retrieval info: USED PORT:
guotient 0
OUTPUT NODEFVAL "guotient [ 131..01" OUTPUT NODEFVAL *remain [31
Retrieval info
PORT:
remain
11 Retrieval info: CONNECT: @acir 0
actr 0 0 0
11 Retrieval info: CONNECT: @clock 0 0 0 0 clock 0 0 0
11 Retrieval info: CONNECT:
@denom
32 0 denom 0 32 0
11 Retrieval info: CONNECT:
@numer
32
numer 0 32
11 Retrieval info: CONNECT:
quotient 0 32
0 @ quotient 0.0 32
11 Retrieval info: CONNECT:
remain 0 0 32 0 @ remaai 0 0 32
11 Retrieval info: GEN FILE:
NORMAL div div_ansigned..
TRUE
11 Reirieval info GEN FILE:
NORMAL
div unsugnggeddd... inc TRUE
div_ansigned cmp TRUE div div_ansigned bsf TRUE div div_ansigned, div se__iniinnd_dbe 11 Reirieval info: GEN FILE: NORMAL div div_unsigned_syye. TRUE
11 Retrieval info: GEN FILE:
TYPE NORMAL
11 Retrieval info: GEN FILE:
TYPE NORMAL
11 Retrieval info: GEN FILE:
TYPE_ NORMAL
TRUE
11 Retrieval info: GEN FILE:
TYPE NORMAL
TRUE
11 Retrieval info: FILE: 1pm
Summarized Problem for The Code
You need to design Verilog module that performs unsigned division two 32-bit numbers. The module should have an asynchronous clear signal (aclr) to reset the division operation, and clock signal (clock) to synchronize the prucess. The module should lake two 32-bil inputs: the divisor (denom) and the dividend (numer). should produce two 32-bit outputs: the quotient (quotient) and the remainder (remain). The module should he optimized should ensure the remainder is always posilive, This Verilog module, named 'div unsignedd". has the interface designed follows:
for speed with an I-stage pipeline
Signal Name Direction Width Description
"aclt' Input 1 Asynchronous clear signal to resel the module
"clock' Input 1 Clock signal synchronize the division operation 'denom' Imput 32 32-hit unsigned divisor input "numer" Input I 32-bit unsigned dividend imput
"quotient" Output 32 32-bit unsigned quotient output
"remain" Output 32 32-bit unsigned remainder output, always positive
E
Prompts
Below shows the prompt for generating instructions given the GitHub codes by DeepSeck- V3,
31

---

Prompt for Generating Questions Given Codes
#ätt demonstration
Please write a description about the code snippet and problem such that the given code snippet exactly the answer to this problem.
[The Code Snippet]
verilog
module top_ (odule(
input ring
input sibrate_mode,
output ringer.
output mator
): assign ringer = ring & -vibrate, mode; assign motor ring & sibrate_mode;
endmodule
[Description]
This Verilog module top-level module that controls the ringer motor of a device. and vibrate mode) and two outputs (ringer and motor). The ring input signal that is used in indicate when the device should be ringing vibrating. The vibrate mode input vibrating. The ringer ourput is a signal that is used to control the ringer of the device. is set to 1 when the device should be ringing and when should not be ringing. The motor output is signal that is used to control the motor of the device. set to when the device should ments to control the ringer and motor outputs. The first assign statement sets the ringer output to the logical AND of the ring when ring input is
four ports: two inputs (ring
signal that indicates whether
device should ringing or
vibrating and when should not he vibrating.
module uses two assign state-
and the logical NOT of the vibrate mode inputs This and vibrate mode input 0, and it will be set to otherwise. The second assign statement sets the motor output to the logical AND of the ring and the vibrate_ mode inputs. This meuns that the motor output will be set to when both the ring module used control of a device bused on the ring and vibrate mode inputs. When the ring input is and the vibrate mode input is O, the ringer output will be to and the motor output will be set to O. When the ring input I and the vibrate mooele input is 1. the ringer output will set to and the motor output will set to [Prohlem]
that the ringer output will be
input and the vibrate mode ipput
ringer and
Suppose you designing circuit control cellphone's ringer vibration motor. Whenever the phone needs ring from an incoming call (input ring). your circuit musi either turn on the ringer (output ringer = 1) or the motor (output motor 1), but not both. If the phone is in vibrate mode (input vibrate mode 1), tumn on the motor. Otherwise, tum on the ringer. Verilog module, named "top_mm module', has interface designed follows: Signal Name Direction Width Description
"ring" Input Indicutes when device should be ringing or vibrating I
"vibrate mode' Input 1 Selects vibrate mode; when high, the device vibrates instead of ringing 'ringer" T Output Controls the device's ringer; high ring. otherwise 'motor' Ourpat Controls the device's 's motor; high to vibrate, low otherwise
demonstration2
Please write a description about the code snippet and problem such that the given code snippet is exactly the answer to this problem.
[The Code Snippet]
verilog
module cop_moduled input elk.
resetn input [1:0] hyteena,
input [15:0] d.
output reg [15:0] q):
always @ posedge clk) begin if (1 rerssnt)).
q 0: else begin
if (byteena[0))
917:0] <= d[7:0];
if (byteena [I])
q[15:8] d[15:8];
end
end endmodule
[Description]
register. has inputs one nutput. The inputs are a clock signal (clk), reset signal (sesetn). 2-b byte enable signal (byteena), and a 16-bit data signal (d). signal is low,
Verilng module top-level module is used to write to
The output
16-bit register (g). The module triggered the register is reset to Otherwise, the data signal written to the register based on the byte enable signal. If the first bit of the byte enable signal is high, the lower 8 bits of the data signal are written to the lower 8 bits of the register, If the second bit of byte [Problem] Create 16 D flip-flops. It's sometimes useful to only modify parts of control whether each hyte of the 16 registers should he written in that cycle. hyteena[1] controls the while bytcena[0] controls the lower byte d[7:0]. resetn a synchronous, active-low reset. All DFFs should be triggered by the positive edge of elk. This Verilog module, named "top_module" module", has the interface designed as follows: Signal Name Diroction Width Description
rising edge of the clock signal. When the
signal high, upper
the signal are written to the upper hits the register.
of flip-fiops. The byte-enable inputs byte d[15:8),
"clk' Input 1 Clock signal "resetn" Imput
Active low reset signal
'bytocna" Input 2 Byte euable signal 'd' Input 16 Data input signal
1911 Outpat 16 16-bit register output
32

---

(continued)
fi* demonstration3
Please write description about the code snippet problem such that the given code snippet is exuctly the answer to this problem.
[The Code Snippet]
verilog
module
top module( input clk
input reset
output reg [3:0] 9):
always @ posedge clk) 10) (reset
q <= 1;
q <= < q+l;
endmodule
[Description]
This top Verilog module is a simple counter that increments its output q by one every clock cycle. It has 3 inputs, clock (cIk),
(reset), and output register (g). The output register 4-bit register. meaning from 0 to The module is triggered on the rising edge of clock signal. When the reset signal is active, the output register set to If the reset signal is not active. the output register is incremented by one. When the output register reaches 10 (1010 binary), it reset to 1. This process is repeated every clock cycle. [Problem]
store values
Make decade counter that counts through 10, inclusive. The input active high synchronous, and should the counter to 1. This Verilng module, named 'top_ module", has the interface designed as follows:
Signal Name Direction Width Deseription
'elk' Input 1 Clock signal that triggers the counter on its rising edge
'resett Input 1 Active-high synchromous reset signal initialize the counterl I I'q" Ourput 4 4-bit register output representing the current count (1-10)
demonstration4 Please write a description about code snippet and problem such that the given code snippet exactly the unswer to this problem. (The Snippet]
verilog
module
top module input elk
input reset
output reg [4:0] 9):
logic [4:0] q_next;
always ((q) begin
q_next q[4:1]:
q_next [4] = =40]]
[ex == 910]:
always @ posedge clk) begin (reset)
5'hl:
else
q < q_next;
endmodule
[Description]
The top module has 3 imputs and output, where the inputs are clk, reset, and ourput is The module hus 2 always blocks transitions defined in the first always block, which
define the state transition of and the logic description. The triggered when q changes. In the first always block, 9 next is assigned with q[4:1]. which is the value q except the LSB bit, Then, g__ext[4] is assigned with q(o). which the LSB bit. Lastly. [_ is xored with q(0). The second always block is tnigeered at positive edge clk. reset active, q assigned with 5'hl. which is inactive, is assigned with nexext which the state transition. The port connections of instantiated modules are shown above. The module takes clk, resct, positive edge of clk. [Problem]
reset value. If
as input. q
output, which assigned with 5'hl reset and next
linear feedback shift register shift register usually with a few XOR gutes produce next state shift reg- ister. A Calois LFSR is one particular arrangement where bit positions with a "tap' are XORed with the nutput bit to produce next value, while bit positions without made to be "maximun-length
be shift. the taps positions are carefully chosen, LFSR A maximum- llngth LFSR of a bits cycles through 2**p states before repeating (the all- state is never reached). Build 5- bit maximal length Galois LFSR with taps bit positions 5 and The active high synchronous reset should reset the LFSR output to I. This Verilog module, named module.. has the interface designed follows:
-enggh
Signal Name Direction Width Description
'olk' Input Clock signal that triggers state transitions rising edges.
I "reset" Input 1 Active-high synchronous reset signal to initialize the L.ER..
'q" Output 5 Current state the LFSR, representing 5-bit value ('00001' to '11111",
33

---

(continued)
fi# demonstrationS
Please write a description about the code snippet and problem such that the given code snippet is exuctly the answer to this problem.
[The Code Snippet]
verilog
module
top_ module
input 199:0] in.
output [98:0] out out_both
output [99:1] aut_any..
output [99:0] out out_different
):
ussign out_ out_both in & 199:111; assign out_any in in119::1]
assign out_different =i0 in 199:1111;
endmadule
Description]
This Verilog module used compare input signals and generate three output signals. The first input signal 100- bit wide vector, and the second input signal is the same vector shifted by one bit. The module has three output signals. out both, out any and out different. The both signal generated by performing bit-wise AND operation between the two input signals. This will result a 99-bit wide vector, where each bit is only if both the corresponding bits the two input signals are 1. The ont signal is generated by performing bit-wise OR operation between the two input signals. This will result
100-bit wide vector, where each signals is 1. The nut different signal generated by performing a bit wise XOR operation hetween the two input signals. This will result
if either the corresponding bits the two input
100-bit wide vector, where euch bit only the corresponding bits of the input signals different. The first hit of the out different signal is generated by performing a hit-wise XOR operation between the first hit of the first input signal and the last bit of the second input signal. [Problem]
You are given bit-bit inpat vector in[99:0]. We want to know some relationships between each bit and its neighbour: (1) out both: Each bit of this output vector should indicate whether both the comesponding input bit and neighbour the left are 'I", For example, out both[98] should indicate if in[98] und in[99] are both 1. Since in[99] has no neighbour to the left, the answer
obvious
don't need to know out both[99].
If (2) out out_any: any: Each bit this output vector should indicate whether any of the conreponning input and its neighbour to the right are I. For example, out any[2] should indicate if either in[2] or in[]] are I. Since in[0] has neighbour to the right,
answer obvinus we # (3) out different: Each bit of this output vector should indicate whether the corresponding input bit is different from its neighbour the left. For example, the vector as wrapping around, so in[99]'s neighbour to the left is in[0].
need know out any([]]
different[99 [98] should indicate in[98] different from in[99]. For this part, treat
This Verilog module, named 'top module'. has the interface designed follows:
Signal Name Direction Width Description
'in' Input 100 100-bit input vector for analyzing bit relationships
'out_ 'out_both" Output I .99 Each bit indicates if both the corresponding input bit and its left neighbor are 'I'
"out Output 19 Fach indicates if either the comesponding input hit its right neighhor '1" 'out_ different Output 100 Each indicutes if the corresponding input bit is different from its left neighbor, circularly
#** Instruction
Please write description about the code snippet and problem such that the given ende snippet is exactly the answer to this problem.
(The Code Snippet]
verillog
(The Given Code Snippet]
NeW Response
Our prompt begins by presenting five distinet demonstrations. Each demonstration first provides description of code snippet, followed by the generation of corresponding problem. We then prompt the model (DeepSeek- V3) to generate a problem similarly based on the given code snippet colored in red. This process mirrors the multi-level summarization mechanism in CodeV [48].
We also show the system prompt we use during training (both SFT and RL) and testing (on bench- marks) as below.
System Prompt for Training and Testing
You are helpful assistant. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within <think> </think> and<answer> <'answer> tags, respectively. i.c., <think reasoning process here </think><answer> answer here <lanswer>. Now the user asks you to write verilog code. After thinking, when you finally reach a conclusion, enclose the final verilog code in verilog A. within <answer> </answer> tags. i.e., <answer> .** "veriloghn module top_ _op_modul(iin, ou); ""aaanswer>.
34

---

Broader Impacts
Through distillation from DeepSeek-RI and reinforcement learning, CodeV-RI-7B even outperforms PrepSeek-R1-6711 on RTLLM v1.1 and RTLLM v2, while outperforming previous Verilog- domaan state-of-the-ar models (typically 15B) by 12~21 % on RTLLM v1.1 and v2. Through these results, our work demonstrates the promising potential of reinforcement learning for improving circuit design.
However, analogous to other code generation models, CodeV-RI V---BB may produce code that misaligns with user intentions or even be misused for unintended purposes. As comprehensively analyzed in broader impact studies [1], such risks include but not limited to:
1. Functional misalignment:
Generated code might superficially satisfy requirements but fail to execute as intended, particularly in safety-critical circuit designs.
2. Security vulnerabilities: The model could inadvertently generate insecure code (e.g., flawed logic backdoors), which poses risks in hardware deployment.
3. Misuse in malicious contexts: Lower barriers to code generation may facilitate the creation of obfuscated harmful designs, especially as model capabilities scale.
Given the potentially severe consequences of such issues in hardware systems, strongly recom- mend that users:
1. Conduct rigorous functional verification and security audits for all generated code.
2. Implement access controls and usage monitoring to mitigate abuse risks.
3. Adopt principle of "human-in--hheloop" oversight, particularly for high-stakes applications.
G Limitations and Future Work
This work has several limitations, and we primarily discuss two key aspects that also define our future direction: (1) The automated testbench generation framework only improve the semantic consistency between code and NL in the probabilistic sense. The synthetic dataset generated by our method both for SFT and RL may still contain small amount of low-quality data, which could potentially impact the model's performance. (2) Collecting data with reasoning processes for SFT requires general reasoning model (e.g.. DeepSeek-RI) which inherently depends the teacher model's reasoning capabilities. This dependency poses greater challenges in specialized domains where the teacher model's performance is suboptimal, as its limitations such contexts may directly impact the quality of the collected data. Besides, this process might be financially costly.
Additionally, from application perspective, it is promising focus on exploring the potential of reasoning LLMs to tackle more complex hardware development tasks beyond RTL code generation in the future, such PPA performance optimization and analog circuit synthesis.
35


Reviews:
[{"review_id": "tsUYDu97Li", "reviewer": "Reviewer_ukvi", "summary": "This paper introduces R2TL, a tool that uses reasoning to generate RTL. The paper goes into rigorous details on how they generate RTL, assess it's quality, and measure the performance of the code generated quality and the code generate performance.", "questions": "+ R2TL versus future reasoners/coder models? Fundamentally, what does R2TL bring to the table beyond SOTA models, and how can it use new models to improve RTL code generation?", "limitations": "N/A.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "This paper addresses an important problem, namely RTL code generation. RTL code generation is challenging because LLMs typically don't have enough examples to \"naturally\" produce RTL, plus there are some precision requirements where an LLM cannot make mistakes in (e.g., clk and reset configuration). This makes it a challenging problem, and the paper attempts to address this via reasoning.\n\nI particularly enjoyed the plethora of different ML-based techniques for this system's level problem. This includes the adaptive DAPO for reducing cost, distillation, and focusing on high quality data over too much data. While not particularly novel, the application here to RTL generation is nice, plus the results are decent. Further, the proofs in the appendix, and the data distribution measurements are well received by this reviewer, and hoping more papers employ such rigor in their evaluation.\n\nWhile not a true weakness: the results aren't mindblowing compared to state-of-the-art methods. So the question then becomes is such a complex approach necessary, or will future reasoning model releases potentially outperform R2TL? I still did appreciate the thorough evaluation, but as a user, would the author recommendation be to use R2TL right now, or wait for Deepseek-R1+N? That said, it seems like future R2TL can benefit from a vanilla DeepSeek reasoner and/or coder model (qwen2.5), so perhaps there is lots of merit to R2TL. Could the authors comment a bit about such a scenario?", "was_revised": true, "final_justification": "Rebuttal helped convince me that my score is justified, and I find this a very good paper and valuable contribution to the community.", "rebuttal": "Thank you sincerely for your thoughtful and detailed feedback on our work. We greatly appreciate your recognition of the importance of the RTL code generation problem, as well as your positive remarks on our adoption of various ML-based techniques. Your inquiries regarding model applications have offered valuable guidance for both the current implementation and future development of R2TL. Addressing your primary concern, we present our perspectives below.\n\n- **Q1:** The results aren't mindblowing compared to state-of-the-art methods. So the question then becomes is such a complex approach necessary, or will future reasoning model releases potentially outperform R2TL? I still did appreciate the thorough evaluation, but as a user, would the author recommendation be to use R2TL right now, or wait for Deepseek-R1+N? That said, it seems like future R2TL can benefit from a vanilla DeepSeek reasoner and/or coder model (qwen2.5), so perhaps there is lots of merit to R2TL. Could the authors comment a bit about such a scenario?\n\nThank you for raising this point. The core value of R2TL lies in its unique advantages that make it a compelling choice for specific use cases.​\n\nFirstly, R2TL boasts significant deployment advantages due to its compact size. With only 7B parameters, it stands in stark contrast to DeepSeek-R1’s 671B parameters, reducing storage requirements by two orders of magnitude. Additionally, its computational demands are approximately an order of magnitude lower than DeepSeek-R1 under the same token budget. Importantly, R2TL is already well-equipped to handle a range of automated design tasks effectively. These characteristics—strong capability paired with high deployment efficiency—make it a viable and excellent choice for local deployment, as it can be set up and run with minimal resource investment, allowing for quick integration into existing workflows.​\n\nSecondly, R2TL represents a generalizable approach that can evolve alongside advancements in general-purpose models, showing the effectiveness of the distillation followed by RL approach in the hardware design domain. Its performance is inherently tied to the capabilities of the models in its training pipeline—both distillation models like DeepSeek-R1 and base models like Qwen2.5-Coder-Instruct. As these models improve, with distillation models generating higher-quality training data and base models providing stronger foundational architectures, R2TL’s performance will strengthen in tandem. This adaptability allows it to leverage future developments in such models, ensuring it remains relevant and powerful as the field progresses.​\n\nThirdly, in terms of user applicability, R2TL is particularly well-suited for professional developers and researchers with specialized needs. While entry-level users and developers may find general-purpose models more accessible via web interfaces or APIs to meet basic requirements, R2TL offers unique value to professionals. Specifically, when these users possess private data or their own base models, they can leverage R2TL's training pipeline to develop dedicated models that are more powerful and better suited to their specific needs. This makes it highly beneficial for those in fields like chip automated design or other AI domains to tackle domain-specific design tasks effectively.\n\nIn summary, R2TL is not only a practical tool for current applications with its deployment-friendly size and efficiency but also a generalizable approach that can grow with advancements in foundational models, making it particularly valuable for professional users requiring independent deployment or training with their unique data.\n\nThank you again for this valuable discussion. We welcome further discussion if you have any questions."}, {"review_id": "RehXw0O6M0", "reviewer": "Reviewer_NGaP", "summary": "The paper introduces ReasoningRTL (R2TL), , a framework to train RTL generation LLM via reinforcement learning with verifiable reward. The framework consists of 3 main contributions: a rule-based testbench generator that performs robust equivalence checking against\ngolden references, a round-trip data synthesis method that creates a high-quality dataset via validating equivalence between original and regenerated code, a two-stage training pipeline for supervised distillation and an efficient RL algorithm (adaptive DAPO). Evaluation shows it surpasses prior state-of-the-art by 12∼21%.", "questions": "- How could the R^2TL be potentially combined with agentic methods? Will it still outperform large size model? e.g, MAGE: A Multi-Agent Engine for Automated RTL Code Generation (DAC 2025), which demonstrates  95% pass rate for VerilogEval dataset.\n- What is the complexity of the RTL code (#tokens) obtained from GitHub? Any case with a very long context length that fails with the 16k context length? \n- How does the rule-based test bench generation can be generalized to larger-scale or complex modules/ hierarchical modules? \n- What could be some potential solutions to further reduce the 2.7% wrong flags?\n- How does the method can be potentially generalized to other HDLs like VHDL, System Verilog?", "limitations": "See weakness", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Applying RLVR to RTL generation domain is a new yet important scope.\n- The rule-based test-generation is novel and achieves 35.5 % fewer false negatives and 62.5 % more injected errors detection in fuzzing tests.\n- The round trip data synthesis method is novel and ensures a high-quality training data.\n- The adaptive DAPO leads to 1.25x speedup for RL\n- The framework shows 12-21% improvement over prior SOTA with a 7B model, even better than 671B DeepSeek-R1 in some testing datasets.\n\nWeaknesses:\n- The rule-based test bench generation may not be good enough for complex or real-world Verilog design, requiring manual maintaining with more expert heuristics.\n- While the method outperforms SOTA on RTLLM (v1.1 and v2), it underperforms on VerilogEval, I believe the RTLLM has fewer tests and is relatively easier than VerilogEval.\n- 3.1k data is small (while demonstrated high-quality), which may affect the generalization and robustness of RL training, even induce overfitting to the dataset distribution. This is not well verified and discussed in the paper.", "was_revised": true, "final_justification": "The paper addressed an important problem of generate high-accuracy RTL code using 7B models, by integrating several key techniques (RLVR, rule-based test generation, data synthesis) in a novel way. The results are promising. The rebuttal addressed my concerns.", "rebuttal": "Thank you for your valuable feedback. We appreciate your recognition of our contributions, including our method and model performance. Your insights on the limitations, including rule-based testbenches’ scalability, VerilogEval performance, training data size and complexity concerns, and agent performance, are crucial and will guide our improvements. We will address your concerns as follows:\n\n- **W1&Q3:** The rule-based testbench generation may require manual maintaining with more expert heuristics for complex or real-world Verilog design. How to generalize it to larger-scale or complex/hierarchical modules?\n\nThank you for pinpointing the challenges of rule-based approaches in complex Verilog verification. Our testbench, built on manual rules, runs fully automatically and works directly for **complex multi-module designs like SHA256 or FFT**, validating DUT-golden code equivalence via consistent I/O checks.\n\nLimitations of our framework include unguaranteed coverage with random I/O, no support for multiple clocks, and certain syntax issues (e.g., macro definitions, Verilog-SystemVerilog reserved word differences). A future direction is to added LLM-generated directed cases to target corner cases missed by random stimuli.\n\n- **W2:** While the method outperforms SOTA on RTLLM, it underperforms on VerilogEval, I believe the RTLLM has fewer tests and is relatively easier than VerilogEval.\n\nWe sincerely thank you for highlighting the performance differences across benchmarks. We emphasize that RTLLM is **harder and more practical** than VerilogEval. Moreover, our model has also achieved **SOTA results among 7B models** on VerilogEval v1-Human (surpassing the previous SOTA model CraftRTL-DS-6.7B by 4.5%) and VerilogEval v2.\n\nIn terms of practicality, VerilogEval [1] explicitly states that \"we do not evaluate the performance of downstream circuit implementations, a gap that is addressed by the work presented in RTLLM\". Therefore, RTLLM is more aligned with real-world applications. The test coverage on RTLLM is strong, with 98.5% average line coverage for v1.1 and 96.0% for v2, which is satisfactory.\n\nRegarding difficulty, results in Tables 1&2 show most models (except Qwen2.5-Coder and RTLCoder-Mistral-7B) perform better on VerilogEval than RTLLM, including our distillation model R2TL-7B-Distill. RTLLM is also harder by code length -- The table below provides the average number of lines and tokens (comments and blank lines removed) for VerilogEval and RTLLM:\n\n| Dataset           | Avg Lines | Avg Tokens |\n| ----------------- | --------- | ---------- |\n| VE1 Machine       | 13.89     | 93.85      |\n| VE1 Human         | 15.82     | 117.35     |\n| VE2 Code Complete | 16.10     | 121.84     |\n| VE2 Spec to RTL   | 16.12     | 122.31     |\n| RTLLM 1.1         | 56.52     | 470.48     |\n| RTLLM 2.0         | 46.30     | 403.80     |\n\nMeanwhile, our dataset's distribution is more consistent with RTLLM than VerilogEval. Details can be found in the answer for Q1 for reviewer oSjb.\n\n[1] Liu, M. et al. Verilogeval: Evaluating large language models for verilog code generation.\n\n[2] Su, H. et al. One embedder, any task: Instruction-finetuned text embeddings.\n\n- **W3:** 3.1k data is small (although high-quality), which may affect the generalization and robustness of RL training, even induce overfitting to the dataset distribution. This is not well verified and discussed in the paper.\n\nWe appreciate your note on the dataset scale. Our model shows strong generalization on unseen benchmark problems, as evidenced by the performance gain from R2TL-7B-Distill (without RL) to R2TL-7B (with RL) on VerilogEval and RTLLM (Tables 1&2). Note we have rigorously de-contaminated our training data against benchmarks to avoid overfitting as shown in Section 2.2.\n\nMoreover, 3.1k is typical for RL in this context. LLM-based RL prioritizes data quality over quantity: Qwen3 [1] utilized 3995 query-verifier pairs during their reasoning RL phase. LIMR [2] filtered 8523 RL data down to 1389 and achieved superior results. Logic-RL [3] enhanced models' logical reasoning capabilities using a mere 900 instances. All these works demonstrate that 3.1k data is sufficient for our RL setup. We will clarify this in the revised paper.\n\n[1] Yang, A. et al. Qwen3 technical report.\n\n[2] Li, X. et al. Limr: Less is more for rl scaling.\n\n[3] Xie, T. et al. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning.\n\n- **Q1:** How could R2TL be potentially combined with agentic methods? Will it still outperform large size model? e.g, MAGE: A Multi-Agent Engine for Automated RTL Code Generation, which demonstrates 95% pass rate for VerilogEval dataset.\n\nThank you for your forward-looking inquiry regarding agentic integration. We note two fundamental distinctions between MAGE and R2TL: First, the MAGE agent uses the golden testbench while R2TL does not. Second, the performance of MAGE strongly depends on prompt engineering and is model-dependent (see table below), which does not align well with the model capacity.\n\nGiven these divergences and computational constraints (MAGE's multi-agent sequential calls limiting parallelization efficiency, as well as the time limitation of rebuttal period), we designed a targeted experiment, using R2TL to replace the RTL generation agent (which best aligns R2TL's ability among the four agents) in MAGE, keeping other agents based on Claude. The test results are shown in the table below.\n\n| Model | Pass@1  |\n| - | - |\n| Deepseek-R1-0528 | 80.77 % |\n| Deepseek-V3-0324 | 88.46 % |\n| GPT-4o-240806 | 76.28 % |\n| Claude-3.5-Sonnet | 95.51 % |\n| R2TL (with claude) | 94.87 % |\n\nThese results demonstrate the potential of R2TL to be applied in the agent workflow, which implies R2TL could develop end-to-end agent capabilities in the future.\n\n- **Q2:** What is the complexity of the RTL code (#tokens) obtained from GitHub? Any case with a very long context length that fails with the 16k context length?\n\nThank you for raising this point. Below are some statistics of our 87k distillation dataset:\n|  | Mean   | Max   | % over 16k tokens |\n| - | - | - | - |\n| RTL code | 537.5  | 3375  | 0.0% |\n| Response (DS-R1 CoT) | 6301.5 | 39927 | 0.5% |\n| Prompt+Response | 6831.3 | 40319 | 0.8% |\n\nDistillation considers (prompt + response) length: around 0.8% of the data exceed the 16k context length and truncated. Meanwhile, the longest RTL code in our dataset is 3375 tokens. Some examples of long code include single round of SHA-256 calculation, CORDIC arctan() implementation, BCD-format 12-hour clock with AM/PM, etc.\n\n- **Q4:** What could be some potential solutions to further reduce the 2.7% wrong flags?\n\nThank you for your concern about this. The 2.7% mismatch in golden-vs-golden comparisons mainly stems from corner syntax errors, like inconsistent variable names, generated by the automated testbench, while the runtime inequivalences only count for **0.3%**. In contrast, the 7.6% inaccuracy for LLM-generated testbenches excludes such test-phase errors. Using a consistent standard, our method’s false negative rate is less than 1/20 of the LLM approach, showing superior stability.\n\nFor the remaining inequivalences, around half involve random numbers, which fall outside our testbench's scope. For the remaining half, removing the `-g2012` compile option works for some cases, and **switching from Icarus Verilog to VCS** can work for most cases. (However, VCS requires license so it is not as convenient as Icarus Verilog.)\n\n- **Q5:** How does the method can be potentially generalized to other HDLs like VHDL, System Verilog?\n\nWe appreciate your question about HDL generalization. Our method is adaptable to other HDLs: we already support SystemVerilog (for VerilogEval) and Verilog (for RTLLM). To extend to other HDLs like VHDL, our five-stage framework (Figure 1) can be adjusted for specific languages:\n\nIn pre-Stage 1 data preparation, we would collect language-specific code snippets (e.g., VHDL modules) from relevant repositories, similar to how we gathered Verilog from GitHub.\n\nFor Stages 3-5, our automated testbench framework would be extended to support the target HDL (mainly syntax adaptation), enabling proper verification during difficulty filtering (Stages 3,5) and equivalence checking (Stage 4).\n\nWith such adjustments, R2TL can generalize to more HDLs beyond Verilog and SystemVerilog.\n\nThank you again for this valuable discussion. Please feel free to ask if you still have any questions."}, {"review_id": "dnScYu0LVJ", "reviewer": "Reviewer_oSjb", "summary": "This paper introduces ReasoningRTL, a Reinforcement Learning with Verifiable Reward (RLVR) framework designed for generating Register-Transfer Level (RTL) hardware descriptions. The proposed framework comprises three main components:\n- A rule-based testbench generator that produces test cases to verify the correctness of RTL code generated by LLMs;\n- A natural language description generator that converts RTL code snippets into natural language descriptions to serve as inputs for model training;\n- A training framework that integrates cold-start SFT and adaptive DAPO to enhance the model's RTL generation capabilities.\n\nThe framework is used to train a 7B-parameter LLM, which is then evaluated on multiple RTL generation benchmarks.", "questions": "N/A", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "trengths:\n+ The paper is generally well-written and well-structured.\n+ The natural language description generator addresses a key bottleneck in RTL generation: the scarcity of high-quality, task-specific training data.\n+ The approach is relatively cost-effective, requiring only 2656 A100-GPU hours for training, which is competitive for LLM development.\n\nWeaknesses:\n- The evaluation results require further clarification. The proposed model shows a notable improvement over existing methods on the RTLLM dataset, yet achieves similar or even worse performance on the VerilogEval benchmark. This raises questions about dataset-specific characteristics and should be more thoroughly discussed. What makes RTLLM particularly amenable to improvement under this framework?\n- Definition 2.1 and Theorem 2.1 are confusing. Given the inherently probabilistic nature of LLMs, the conditions in Definition 2.1 are unlikely to hold in practice, which undermines the applicability and significance of Theorem 2.1. Moreover, they appear to have limited relevance to the rest of the paper.\n- The natural language description generator component would benefit from a deeper analysis. While the appendix includes some illustrative examples, they are limited to short RTL code snippets. In practice, generating natural language descriptions for complex RTL modules, especially those involving non-trivial sequential logic, is a challenging task. Without access to simulation traces, inferring sequential behavior from code alone is difficult even for human experts. This raises concerns about the generator's ability to generalize to realistic and more complex RTL designs.", "was_revised": true, "final_justification": "The authors have addressed my earlier concerns regarding the experimental results. However, I remain unconvinced about the relevance of Definition 2.1 and Theorem 2.1 to the rest of the paper. In particular, the assumption of determinism in generating programs and natural language descriptions seems misaligned with the nature of RTL design, where multiple valid representations and descriptions naturally exist. I have therefore adjusted my score to 4.", "rebuttal": "We sincerely thank you for your comprehensive feedback, particularly for your appreciation of our writing, training data generation process, and training framework. For your concerns on our work's cross-benchmark performance variations, theoretical foundations, and NL generation for complex modules, we would like to respond as follows:\n\n- **Q1:** The evaluation results require further clarification. The proposed model shows a notable improvement over existing methods on the RTLLM dataset, yet achieves similar or even worse performance on the VerilogEval benchmark. This raises questions about dataset-specific characteristics and should be more thoroughly discussed. What makes RTLLM particularly amenable to improvement under this framework?\n\nThank you for your focus on dataset-specific characteristics. Given that RTLLM's performance gains stem mainly from reinforcement learning, we analyzed distribution differences between our RL dataset and the two benchmarks. We run the instructer-embedding model [1] for the golden code in our RL dataset, RTLLM (v2), and VerilogEval (v2 spec-to-RTL), then generated a t-SNE distribution plot [2]. This plot revealed our RL dataset aligns closely with RTLLM but diverges from VerilogEval (to be added in revision). Below we also show cosine similarity and Euclidean distance metrics for the **embedding centroids** between our RL dataset and the benchmarks:\n\n|                    | RTLLM v2 | VerilogEval v2 |\n| ------------------ | -------- | -------------- |\n| Euclidean Distance | 0.1150   | 0.2903         |\n| Cosine Similarity  | 0.9926   | 0.9547         |\n\nOur RL dataset’s embedding centroid shows significantly smaller Euclidean distance and closer-to-1 (maximum value) cosine similarity with RTLLM than with VerilogEval. Importantly, this reflects only **embedding centroid** relationships, not data homogenization or overfitting to RTLLM.\n\nAdditionally, we conducted a detailed case study across both datasets and identified one reason is VerilogEval’s heavier use of table/graph-based problems, where our model underperforms significantly. Below is a comparison of our model's accuracy against Deepseek-r1 on these problem types, including their ratios within benchmarks:\n\n|                           | Tables/Graphs (Ours) | Tables/Graphs (DS-R1) |\n| ------------------------- | -------------------- | --------------------- |\n| VerilogEval v2 (Ratio)    | 29.49%               | 29.49%                |\n| VerilogEval v2 (Accuracy) | 55.43%               | 78.04%                |\n| RTLLM v2 (Ratio)          | 0%                   | 0%                    |\n| RTLLM v2 (Accuracy)       | N/A                  | N/A                   |\n\nThis points to a key improvement direction: incorporating more table/graph-specific instruction-response pairs (e.g., KMap, FSM, waveform data as in CraftRTL [3]) into our training dataset.\n\nFinally, we should mention that RTLLM is more practical than VerilogEval since VerilogEval clearly states \"we do not evaluate the performance of downstream circuit implementations\". The similarity between our dataset and RTLLM shows the practicality of our dataset. In addition, we have gone through rigorous data decontamination against these benchmarks by Rouge-L score during data collection (in section 2.2).\n\n[1] Su, H., Shi, W., Kasai, J., Wang, Y., Hu, Y., Ostendorf, M., ... & Yu, T. (2022). One embedder, any task: Instruction-finetuned text embeddings. arXiv preprint arXiv:2212.09741.\n\n[2] Maaten, L. V. D., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of machine learning research, 9(Nov), 2579-2605.\n\n[3] Liu, M., Tsai, Y. D., Zhou, W., & Ren, H. (2024). Craftrtl: High-quality synthetic data generation for verilog code models with correct-by-construction non-textual representations and targeted code repair. arXiv preprint arXiv:2409.12993.\n\n\n\n- **Q2:** Definition 2.1 and Theorem 2.1 are confusing. Given the inherently probabilistic nature of LLMs, the conditions in Definition 2.1 are unlikely to hold in practice, which undermines the applicability and significance of Theorem 2.1. Moreover, they appear to have limited relevance to the rest of the paper.\n\nWe sincerely thank you for your thoughtful critique regarding our theoretical framework.\n\nAs for the stringency of Theorem 2.1's conditions, although LLMs behave probabilistically in text, they can be relatively deterministic when implementing the functionality. For instance, given the NL prompt \"Generate a Python implementation of the quicksort algorithm,\" the specific code text produced might vary slightly (e.g., in variable naming or comment placement), but the core functionality $f$ (i.e., a correctly working quicksort algorithm in Python) remains highly consistent. Notably, the stronger the model's capability and the more specific the NL description, the more deterministic this $L \\rightarrow F$ (NL->code) mapping becomes. This logic also applies in the reverse direction, where generating an NL description ($l$) from a given code ($f$) can exhibit similar determinism under sufficient model competence. We will clarify this in our revised paper.\n\nDefinition 2.1 and Theorem 2.1 are relevant to the rest of the paper based on the above insights. **They provide a theoretical foundation for understanding why the round-trip data synthesis process can enhance data quality**: as model capacity approaches the assumptions outlined in Definition 2.1 (e.g., more reliable bidirectional mappings), the outputs of the synthesis process align more closely with the guarantees in Theorem 2.1, resulting in higher-quality data. Specifically, the deterministic equivalence of generated code and source code will guarantee the correctness of both problem summarization and code generation. Our experimental results further validate the reasonableness of this theoretical framework.\n\nIn conclusion, these definitions and theorems offer critical insights from an information-theoretic perspective, elucidating the underlying mechanisms through which round-trip synthesis reduces noise and strengthens the consistency between input and output spaces—ultimately justifying its effectiveness in improving data quality.\n\n\n\n- **Q3:** The natural language description generator component would benefit from a deeper analysis. While the appendix includes some illustrative examples, they are limited to short RTL code snippets. In practice, generating natural language descriptions for complex RTL modules, especially those involving non-trivial sequential logic, is a challenging task. Without access to simulation traces, inferring sequential behavior from code alone is difficult even for human experts. This raises concerns about the generator's ability to generalize to realistic and more complex RTL designs.\n\nWe sincerely thank you for your insightful perspective on natural language generation for complex RTL. We would like to engage with this perspective through the following technical considerations:\n\nOur dataset contains significantly more complex code than the limited examples included in the appendix. This underscores the capability of our natural language description generator: it can effectively handle such complex code, as evidenced by the high-performance model we trained using the dataset—composed entirely of (problem, code) pairs generated by this generator. Notably, our model can solve asyn_fifo (147 lines of golden code) and float_multi (165 lines) in RTLLM v2. Below is a table comparing the average code length (in tokens) in our dataset with that of the benchmarks (RTLLM v2 and VerilogEval v2 - Spec to RTL).\n\n|          | SFT dataset | RTLLM v2 | VerilogEval v2 |\n| -------- | ----------- | -------- | -------------- |\n| RTL code | 478.50      | 403.80   | 122.31         |\n| Prompt   | 529.81      | 348.82   | 219.94         |\n\nFrom the table, the average code length in our dataset is longer than that in RTLLM v2 and VerilogEval v2. Besides, the longest code in our SFT dataset is **over 3000 tokens**, and those can pass the round-trip equivalence checking in our RL dataset include relatively complex ones like **single round of SHA-256 calculation**.\n\nWe also acknowledge that, although we have trained a high-performance model using our workflow involving this natural language description generator, we have room to further improve this process. For example, we can add mechanisms like multi-turn refinement for code summarization with RTL simulator, code parser or other tools.\n\nThank you again for your valuable feedback. Should our responses adequately address your concerns, we would be grateful for your consideration in recognizing these improvements in your final assessment. If any questions remain, we welcome further discussion."}]

Author rebuttal:

Meta-review: The paper deals with training language models to generate register-transfer level (RTL) code, an interesting and important application in electronic design automation. Broadly it centers on ways to adapt the reinforcement learning with verified rewards (RLVR) framework to this setting, which lacks a verifier and is data-scarce. The paper proposes an automated testbench generation method based on insight into the problem domain (e.g., using circuit-structure analysis and simulations), as well as a round-trip data synthesis method.

The reviewers were all positive, with two accepts and one borderline accept. The problem space is interesting, the method has promising results, and the techniques are interesting and a nice fit for the problem. A remaining concern with one reviewer was the relevance of Definition 2.1 and Theorem 2.1, which should be clarified in the final version. Otherwise the reviewers remain quite positive about this paper. I agree and think that it would be a nice addition to NeurIPS. I recommend acceptance.
Decision: Accept (poster)