QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation
Authors: Yang Zhang
Rui Zhang
Jiaming Guo
Huang Lei
Di Huang
Yunpu Zhao
Shuyao Cheng
Pengwei Jin
Chongxiao Li
Zidong Du
Xing Hu
Qi Guo
Yunji Chen
Venue: NeurIPS 2025 poster ()
Keywords: Verilog Code Generation
Code Generation
LLM
Primary area: deep_learning

Abstract: The remarkable progress of Large Language Models (LLMs) presents promising opportunities for Verilog code generation which is significantly important for automated circuit design. The lacking of meaningful functional rewards hinders the preference optimization based on Reinforcement Learning (RL) for producing functionally correct Verilog code. In this paper, we propose Signal-Aware Learning for Verilog code generation (QiMeng-SALV) by leveraging code segments of functionally correct output signal to optimize RL training. Considering Verilog code specifies the structural interconnection of hardware gates and wires so that different output signals are independent, the key insight of QiMeng-SALV is to extract verified signal-aware implementations in partially incorrect modules, so as to enhance the extraction of meaningful functional rewards. Roughly, we verify the functional correctness of signals in generated module by comparing with that of reference module in the training data. Then abstract syntax tree (AST) is employed to identify signal-aware code segments which can provide meaningful functional rewards from erroneous modules. Finally, we introduce signal-aware DPO which is optimized on the correct signal-level code segments, thereby preventing noise and interference from incorrect signals.
The proposed QiMeng-SALV underscores the paradigm shift from conventional module-level to fine-grained signal-level optimization in Verilog code generation, addressing the issue of insufficient functional rewards.
Experiments demonstrate that our method achieves state-of-the-art performance on VerilogEval and RTLLM, with a 7B parameter model matching the performance of the DeepSeek v3 671B model and significantly outperforming the leading open-source model CodeV trained on the same dataset.

QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation
Yang Zhang 1.2
Rui Zhang'
Jiaming Guo
Lei Huang
Di Huang'
Yunpu Zhao!. 1.3
Shuyao Cheng'
Pengwei Jin
Chongxiao Lj
Zidong Du
Xing Hul
Qi Guo
Yunji Chen 122*
State Key Lab of Processors, Institute of Computing Technology, CAS University of Chinese Academy of Sciences University of Science and Technology of China
https: www.iippr....bbb.... 10/ Lio//iienn-SALV/
Abstract
The remarkable progress of Large Language Models (LLMs) presents promising opportunities for Verilog code generation which significantly important for automated circuit design. The lacking of meaningful functional rewards hinders the preference optimization based on Reinforcement Learning (RL. for producing
functionally correct Verilog code. In this paper, we propose Signal-Aware Learning for Verilog code generation (QiMeng-SALV) by leveraging code segments functionally correct output signal to optimize RL training. Considering Verilog code specifies the structural interconnection of hardware gates and wires so that different output signals independent, the key insight of SaMeng-SAII, is to extract verified signal-aware implementations in partially incorrect modules, to enhance the extraction of meaningful functional rewards. Roughly, verify the functional correctness signals in generated module by comparing with that of reference module in the training data. Then abstract syntax tree (AST) employed to identify signal-aware code segments which can provide meaningful functional rewards from erroneous modules. Finally, introduce signal-aware DPO which is optimized on the correct signal-level code segments, thereby preventing noise and interference from incorrect signals. The proposed QiMeng-SALY underscores the paradigm shift from conventional module-level to fine- ene-grained signal-Ievel optimization in Verilog code generation, addressing the issue of insufficient functional rewards. Experiments demonstrate that our method achieves state-of-the-ee the--rrtt performance on VerilogEval and RTLLM, with a 7B parameter model matching the performance of the DeepSeek v3 671B model and significantly outperforming the leading open-source model Code trained on the same dataset. Our code is available at
SALV
Introduction
Circuit design is inherently complex and time-consuming, particularly with respect to Hardware Description Language (HDL) such as Verilog. Automating the HDL coding process paramount importance as it significantly enhances the efficiency of circuit design. The remarkable progress of
"Corresponding author. Contact: (zhangyang22$2.cy)) @ @ict.ac.cn
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Hoddacop
Design combinational Verilog module decocles 16-bit keyboard scan codes to activate 's' key (16h001c) 'd' key (1610023) directional output signahs, all other inputs producing inactive low outputs.
akwaps
sooe
Made
inoddlll top module impur [15x0) scode,
top, poodule
sop, module
input [15:0)
input [15:0] scode.
ourput reg output reg
oulput rog
outpul nog
outgut
output reg
always (scade) begin
always 68 (scode) begin
always @ (spodi) begin
X
rbb:
X
Uibl;
d lbb;;
$
di L'bo
d+ Tb0;
(scode)
caso (scode)
<aee (scode)
10
The00Te: 175;
10
16'he00te: 181;
10
hell an 1b1:
The0023: d- 1b1;
11
h0'h00233 161:
11
16 16'h00233 d+ lbl;
12
endrese
endesse
12
endiese
13
cnd
13
14
endmodule
endmochile
14
Top module
output signal
output signal
Due to Verilog's 's characteristics of relatively independent and parallel signals, we can extract the code implementation of a specific signal through AST. Here there are two output signals, we can separately extract their related code implementations. Among them, the implementation of signal is incorrect while signal d is correctly implemented. We can utilize the code implementation of the correct signal d to provide functional correctness rewards for RL.
Figure 1:
Large Language Models (LLMs) in program code generation, such Python, presents promising opportunities for Verilog code generation. To equip LLMs with superior code generation capabilities, post-training procedures are essential, primarily comprising Supervised Fine-Tuning (SFT) and preference-basee Reinforcement Learning (RL). SFT provides foundational programming language syntax comprehension and domain-specti knowledge. while RL optimization techniques, like Proximal Policy Optimization (PPO) [. Group Relative Policy Optimization (GRPO) (2] and Direct Preference Optimization (DPO) 3, enhance the model's 's ability produce functionally correct code.
Currently, most Verilog code generation methods primarily focus on the SFT stage, particularly through dataset preparation [ 5 6). Few methods (7 8 employ RL through using code structure similarity with reference code as the reward. However, relying solely on code structure similarity as reward has inherent limitations, since module may has many different Verilog implementations which exhibit structural differences but are functionally correct. Thus, functional correctness as the reward in RL, which requires the model to sample functionally correct module implementations to provide the reward. However, the lack of high-quality SFT training data frequently results in inadequate initial code generation capability, obstructing the acquisition of meaningful functional rewards and consequently impairing RL optimization performance. Thus, enhancing the availability of actionable functional rewards to facilitate effective RL optimization remains an open research challenge.
more reasonable to use
In contrast to high-level program language that defines sequential execution behavior, Verilog code specifies the structural interconnection of hardware gates and wires, granting inherent independence between different output signals. This characteristic enables scenarios where individual signal implementations remain functionally correct despite errors in the overall module implementation. As shown in Fig [ the faulty implementation of the signal results in total malfunction of the top module, whereas the signal d maintains proper functionality. Due to Verilog's inherent signal independence easily extract certain signal implementation from the top module through abstract syntax tree (AST). Such partially valid signal-level implementations can be leveraged within RL frameworks to derive meaningful functional rewards. thereby expanding the effective sample space for RL optimization.
and parallel processing nature,
Based on the above analysis, in this work, we propose Signal-Aware Learning for Verilog code generation (QiMeng-SAi.y by leveraging code segments of functionally correct output signal to optimize RL training. The key insight of MiMeng-SALY is to extract verified signal-aware

---

implementations in partially incorrect modules, so as to enhance the extraction of meaningful functional rewards. Specifically. to verify the functional correctness of signals, some random test inputs are generated to compare signal-level outputs between generated module and reference module in the training data. Additionally, abstract syntax tree (AST) analysis employed to precisely identify preferred and dispreferred code segments, thus these partially correct signal-level implementation can be utilized to generate meaningful functional rewards. Finally, to enable the model to learn correct signal implementations from erroneous modules, we introduce signal-aware DPO which is based on DPO, a method renowned for its computational efficiency, rapid convergence, and strong performance. Unlike standard DPO that computes probabilities for all module code tokens, signal- aware DPO exclusively calculates token probabilities for the correct signal-level code segments from the preferred and dispreferred samples, thereby preventing noise and interference from incorrect signals during RI training. Crucially, the proposed DiMeng-SALV expands the effective training dataset by incorporating all modules containing any valid signal implementations, regardless of overall module correctness. This represents a fundamental shift from conventional module-level to fine-grained signal-level optimization in RL-based Verilog code generation, addressing the issue of insufficient functional rewards in RL training. To the best of our knowledge, this is the first fine-grained signal-Ievel reinforcement learning algorithm designed for Verilog code generation.
Comprehensive evaluations demonstrate that J.Menng-AALL. establishes new state-of- the art results on both VerilogEval [0] and RTLLM benchmarks [2]. The 7B 7--parameter model achieves remarkable 62.6% pass @ I and 75.1% 5 accuracy on RTLLM v1.1, not only matching the performance of the 671B Bararmretterr DeepSeck- V3 [3). but also delivering substantial improvements over CodeV [4), the leading open-source model of comparable scale. Ablation studies further validate the substantial impact of filtering incorrect signal segments, confirming the efficayy of QiMeng-SALL's design.
2 Related Work
2.1 RTL code generation
Recent years have witnessed remarkable achievements of Large Language Models (LLMs) in software code generation tasks, prompting both academia and industry actively investigate their potential applications in Hardware Description Language (HDL) code generation [14). This novel research frontier has achieved significant progress, unveiling promising paradigms for hardware design automation.
Instruction tuning has emerged as a pivotal methodology for enhancing LLMs' task adaptability, owing to its methodological simplicity and demonstrated performance gains 4. This technique shows broad application prospects for automated HDL generation. However, compared to software programming languages (such as Python, C++), the domain-specific expertise of hardware design languages and the scarcity of high-quality training data result in current LLMs' performance in HDL generation tasks being significantly inferior to their performance in software programming languages 15). Therefore, many works focus the construction of hardware language datasets 66,5 and employ Supervised Fine-Tuning (SFT) strategies to enhance LLMs' HDL generation capability. However, the heterogeneous quality of training data, whether scraped from public sources or synthesized by advanced ILMs, remains a critical bottleneck, compromising both validity and reliability. This has spurred innovative approaches incorporating verification frameworks and feedback mechanisms during model training/inference to overcome performance bottlenecks and improve RTL generation quality [16 [4
2.2 RL Training for LLM
The effectiveness of supervised fine-tuning (SFT) in hardware description languages is fundamentally constrained by data quality and diversity, particularly in specialized domains where conventional approaches often fail to meet professional requirements/ [8]. This limitation has driven interest in reinforcement learning (RL) techniques that can enhance model capabilities beyond what SET alone
achieve.
Reinforcement Learning from Human Feedhack (RLHF) 19 is among the earliest and most impactful approaches for aligning language models with human intent with Proximal Policy Optimization [
3

---

has focused on developing more efficient alternatives, with Direct Preference Optimization ( emerging as particularly impactful due to its simplicity and stable training process. The recent Group Relative Policy Optimization (GRPO) approach [ further advanced this direction by eliminating the need for explicit reward modeling, demonstrating particular success in RL-based post-training optimization for higher-performantt L.MSs.
Subsequent work 20
21 3
22
23
In the domain of Verilog generation, current RL-based training approaches face two key challenges: (I) highly rely on structural similarity metrics 77] g that cannot properly evaluate functional equiv- alence between different implementations, and (2) the scarcity of high-quality SFT data limits the initial model capability needed for effective RL optimization. While recent work has explored various reward formulations [24] 25 7 and feedback mechanisms (26] 22] 17] these approaches often intro- duce evaluation biases, limiting their reliability in assessing functional correctness. VeriPrefer [28] mitigates such biases by employing module-level functional rewards during reinforcement learning. Our approach differs fundamentally: we adopt signal-level functional rewards, allowing partially correct modules to still provide valuable feedback during training when they contain correctly im- plemented signals. Consequently, our method delivers denser and more informative reward signals, leading to superior reinforcement learning performance.
3 Method
3.1 Framework Overview
In this work, we propose an innovative training methodology named QiMeng-SALY, which designed to leverage correct signal-aware implementations within erroneous module codes for Verilog code generation, We start from a naive code generator, which is typically fine-tuned using a Verilog training dataset D (I, } based
veneral-purposs. LLM, where (I, y) is the description-code pair. The naive code generator samples & candidate module code implementations given each module design prompt Our primary objective is to extract and learn correct signal-aware implementations from these candidates for improving RL optimization. Here we employ DPO as our RL optimization approach, a technique widely recognized for its computational efficiency, fast convergence, and robust performance.
= {11, 1/2 ..... M/
Our training methodology comprises three stages: 1) Signal-aware Verification: By generating random input signals and comparing the output signals between the generated module and the ref- erence module in the training set, we identify correct output signals to obtain preference dataset P = [w St, e) where you and y represent preferred and dispreferred module code respectively. represent the contrast signal which is correct in Yor and incorrect in yt. 2) Signal-aware Code Extraction: Leveraging abstract syntax tree (AST) analysis, we establish signal dependency graphs and isolate relevant preferred and dispreferred code segments (S SF) corresponding to the contrast signals e from yu and y/. respectively. 3) Signal-aware DPO training: By computing token probabil- ities only for the preferred and dispreferred code segments related to contrast signals, we employ DPO to enable the model to learn correct signal implementations, even when the entire module implementation erroneous. The following sections provide detailed explanations of each stage.
3.2 Signal-aware Verification
Enabling RL to train models that generate functionally correct code fundamentally requires functional reward derived from verified implementations. This makes functional correctness verification critical challenge when applying RL to LLM-based code generation, especially for hardware description languages such Verilog. Conventional verification methods depend on testbenches to assess module functional correctness, but the lack of such testbenches in training datasets substantially restricts the capacity to provide meaningful correctness feedback during RL training.
Our approach addresses this limitation through an automated signal verification process for generating functional reward. Considering the functional reward is provided by preference dataset in DPO, we therefore establish the preference dataset based on automated functional verification by three steps: 1) generating comprehensive random input stimuli, 2) verifying generated signals by comparison with reference modules, 3) collecting preference dataset from the verifying results of generated signals, as depicted in Fig (b).

---

(o) Franework
Ref-Code
Code-1
Prompt
Verification
Signal-aa
Signal- L-aware Code Extraction
DPO Training
Segnaa swarr
Code-2
Coder...
Cali
Call
Calll
Calll
SAL-V
X
X
Naive Code
Code-k
Generalor
(b) Signal-awer Verifiontioo
Extraction
Code Snappet
AST
Signal Dependence
Bignal
Sunalatee
MUTU
outb
BaFCo
Code-1
Codlo
Teatodel
X
faxSode3
Figure 2: Overview of the QiMeng- SALV framework. a) The proposed QiMeng-SAte SALV comprises three stage: signal-aware verification, signal-aware code extraction, and signal-aware DPO training. b) In signal-aware verification stage, verification is performed by analyzing output signal discrepancies between generated modules and their reference counterparts, allowing precise identification of correctly functioning output signals. c) In signal-aware code extraction stage, AST parsing reveals critical dependencies between output signals and intermediate signals to obtain relevant preferred and dispreferred code segments pertinent to the contrastive signals.
First, we use Yosys [29 to analyze the reference module head, automatically generating N sets of input signals equal time intervals. Second, the input signals simultaneously applied to the reference module and generated modules E = {3/1 1/2 3/s J- By comparing the output signals pro- duced by the generated modules with the corresponding output signals from the reference module, can verify the correctness of generated modules, and obtain the pairs {M1 c1). (3/2, o2), (yke cx)}. where G, is the correct output signals for generated module y;. Notably, only output signals exhibiting complete matching across all N sets of input signals are verified as correct, while any discrepancies result in classification erroneous signals. Third, to select preference samples suitable for DPO training, we choose sample pairs that there exists a set of contrast signals that is correct in the preferred sample but incorrect in the dispreferred sample. This can be formally expressed as:
P (g Vt, c) 3c ow $ Ct:
(I)
where denotes the preference dataset, e represents the contrast signals, Gw and oj represent the correct signals in the preferred sample you and dispreferred sample y respectively.
3.3 Signal-aware Code Extraction
In RL-based code generation, there exist two typical feedback mechanisms: outcome feedback and process feedback. Compared to outcome feedback which only indicates whether the final output is correct or not, process feedback can precisely identify which parts of the code implementation are correct, thereby providing more dense reward for the specific code implementation corresponding to the output signal and adopt the RI. method based on process feedback of great significance for code generation.
effective learning. Therefore, by extracting
In contrast to program code, which sequentially describes execution logic, Verilog code characterizes the interconnection between wires and gates. This indicates that signal-describing code blocks are largely self-contained, allowing individual signal implementations to maintain functional accuracy when the broader module contains errors. These partially correct signal-level implementations can utilized in RL frameworks to generate meaningful functional rewards, thus increasing the viable sample space for RL optimization.
Based on the above analysis, in the signal-aware code extraction stage, for given target signals, we parse the Abstract Syntax Tree (AST) to establish dependency relationships between signals, and
5

---

then extract the complete code segment corresponding to the target signals, as depicted in Fig 2 (c). Specifically, given the module code of the target signal, we use Yosys (29 to parse the module code and obtain the AST. Subsequently, we analyze the AST to derive the dependency relationships between all the output signals and intermediate signals defined in the module code, forming a signal topology graph. By analyzing this topology graph and performing backward traversal from the target output signal leaf nodes, we obtain the dependent signals of the target signal. Based on the locations of these dependent signals in the AST, we retain their corresponding code segments, thereby obtaining the complete code segment of the target signal. To facilitate the model's learning by using code segment of contrast signal during DPO training, given the contrast signals c, isolates the relevant code segment S from the preferred sample Yeu Meanwhile, to enable comparison with the preferred sample, we also extract the code segment Si of contrast signal from the dispreferred sample yl-
3.4 Signal-aware DPO Training
In this stage, we employ the preferred and dispreferred code segments related to contrast signals in DPO training to strengthen the model's ability of generating functional correct signal implementations. Standard DPO approaches uniformly increase the likelihood of all tokens preferred code samples while suppressing all tokens in dispreferred samples. This paradigm inherently assumes the absolute correctness of the preferred samples, which often violated in practice due to suboptimal SFT models. Because of the limited training data of Verilog code implementations during the SFT stage, the SFT models frequently fail to generate completely accurate code samples. Our key insight recognizes that while generating completely accurate modules remains challenging, we can still leverage partially code segments of functionally correct output signal from incorrect modules to optimize DPO training.
Therefore, we introduce signal-aware DPO, an improved DPO algorithm for Verilog code generation, which increases the probability of correct signal-relate code in preferred samples while decreasing the probability of erroneous signals in dispreferred samples. To achieve this, during loss computation, we only calculate the token probabilities for the contrasting signal-related code segments in both preferred and dispreferred samples, while ignoring the probabilities of other code segment tokens. Formally expressed as:
[( mTrt) E 85 ST)
log
Σ log
No(Uh (t. (Mr (o..ct S.
Σ log Te (esss Beeluel
(2)
Bu St.
where you and yl represents the preferred and dispreferred sample respectively. S denotes code segments related to contrast signals in preferred samples, while S represents the same signal related segments in dispreferred samples. To is the policy model to be optimized and Tet is the reference model used for regularizing To with Kullback-Leible divergence and 5 is constant to control the degree of regularization. Through the objective function in Eq. (2), signal-aware DPO shift DPO's learning focus from entire modules to individual signals, refining the learning granularity and increasing the quantity of functional correctness rewards in DPO, thereby enhancing optimization effectiveness.
4 Experiment
4.1 Experiment Setup
Datasets Our training dataset is sourced from CodeV 44, consisting of 165k samples obtained by crawling all publicly available Verilog module code on GitHub. However, we observed that a subset of these modules are not syntactically valid, prompting us to filter out such cases and retain a cleaned dataset of 135k samples. The 135k dataset is initially employed to fine-tune the canrrol-purpoee code generation model Qwen2.5 Coder Instruct [30], yielding a naive code generator. For every prompt in the 135k dataset, this generator produces 5 candidate module codes, while the corresponding module codes in the dataset serve as reference codes to validate signal correctness during Signal-aware Verification.
Training Settings In training, we perform full-parameter fine-tuning for epochs during the SFT, followed by about 1 epoch (7000 steps) of LoR A-baeedadd [11 fine-tuning in the Signal-aware DPO. Optimization is carried out using the Adam (32 optimizer with a cosine annealing learning rate schedule, where the initial learning rates are set to 1.0e-5 for SFT and 6.0e-6 for Signal-aware DPO.

---

Table 1: Main Results on VerilogEvalI.0 and VerilogEval2.0 Benchmarks
VerilogEvalL.0
VerilogEval2.0 (%)
Type
Model
Michine
Human
Speeification
Completion
Passee1
Prssett 10
Pass 6t
Pascot
Pass 10
T=U
T=t.8
T=D
T=U
GPT-3.5
General Model
GPT-4o
GPT-4
67.7
60.0
70.6
75.5
73.5
77.2
43.5
60.1
55.8
71.4
74.5
58.9
32.0
62.5
31.7
61.4
59.0
12.3
56.1
41.6
Deepseek
671B
77.6
86.2
87.4
70.7
77.4
78.5
68.8
66.9
68.0
66.1
General Code
Model
Dernweek
CodeQwen1.5
7B
46.3
$4.9
56.4
56.8
22.5
26.
34.9
2X.
21.7
1.9
15.0
19.5
21.5
25.0
29.3
17.9
Qwen2.5 Coder Instract
7B
50.1
66.5
70,9
22.9
36.0
39,5
23,0
22.0
30.1
25.6
RTLCuler
$3.4
36.8
CodeViQwentl.
7B
77.6
88.2
90.7
52.7
62.5
67.3
7.7
8.4
50.0
45.7
Serilog-Specilic
CodeViQwen2.5
7B
77.3
87.9
90.1
57.9
66.7
69.7
14.8
37.4
58.3
49.9
Model
Origen
74.1
$2.4
85.7
60.1
64.2
49.3
46.8
49.3
47.2
VeriScek
VeriPreffer
7B
72.7
85.8
49.7
62.3
55.7
51.3
QiMeng SALV (Ouss
7B
81.4
88.6
90.8
60.4
68.6
71.2
$7.1
56.2
62.2
58.8
The training configuration employs global batch size of 64 and maximum sequence length of 2048 tokens.
Metric Following previous work M. the pass metric employed evaluate model perfor- mance, estimating the probability that at least one correct solution is generated within & independent attempts for each problem:
pass @k == Eproblems
(3)
Here, n 2 l denotes the total number of independent solution attempts per problem instance, while c corresponds to the count of functionally correct solutions among these trials.
Benchmark We conduct comprehensive evaluations on the VerilogEval (including VerilogEvall. 1.
[9] and VerilogEEvl12.0 (T0) and RTLLM benchmarks (including RTLLM v1.1 11 and RTLLM
v2.0 (12).
VerilogEvalI.0 is divided into two subtasks: Machine and Human. The Machine subset contains 143 Verilog design problems, with each prompt generated by LLMs; the Human subset comprises 156 Verilog design problems, with each prompt manually crafted. VerilogEval2.0 revisit some limitaions of VerilogEvall. O [9] and support specification-to-RTT tasks in addition to the original code completion task. Since VerilogEval-macee world code generation problem, VerilogEval2..0 only evaluate models against VerilogEval-huus -human to highlight the most useful LLM evaluation.
can be overly descriptive compared to real-
RTLLM v1.1 [1I incorporates 29 distinct Verilog code generation tasks categorized into Arith- metic and Logic domains. Each task specification provides complete interface definitions (module ports) along with detailed functional requirements. RTLLM v2.0 expands from the 29 Verilog designs in RTLLM v1.1 to 50, covering four categories of tasks: Arithmetic, Control, Memory, and Miscellaneous.
Our experimental setup employs a sampling configuration n=20 generations per prompt. We evaluate the model generated module code using pass 1, @5, and pass @ 10 metries. The pass@) metric specifically quantifies solution accuracy on demonstrably solvable problems, reflecting the model's 's consistency and stability in generating correct implementations. In contrast, pass @ 10 assesses the model's overall problem-solving capacity by measuring its ability to produce at least one valid solution within twenty attempts. thereby characterizing the breadth of its code generation capability.
Following previous practices [16], conduct tests at temperature settings of 0.2, 0.5, and 0.8, and report the highest results on VerilogEval1 0 and RTLLM benchmark. According to the VerilogEval2.00 paper [10], VerilogEval2.0 only reports pass @1 results under low-temperature settings (T=0.0,
top 0=0.1,, n=1) and -egh-temperature settings (T=0.8, top_p=0.95, n=20) using nucleus sampling Notably, pass @5 and pass 10 scores are excluded from the evaluation.
33
Baseline Methods In our experimental evaluation, we conduct comprehensive comparison between proposed method and several baseline approaches, categorized into three groups: I)
7

---

Table 2: Main Results on RTLLM v1.1 and RTLLM v2.0 Benchmarks
RTLLM v1.1 (%)
RTLLM v2.0 (%)
Type
Model
Pass@
Pass@5
Pass 10
Pass@1
Pass@
Pass 10
GPT-3.5
28.3
36.9
41.4
34.4
49.8
52.1
Foundation
General Model
GPT-4o
41.7
65.9
56.5
70.3
75.2
Deepsock v3
671B
62.0
72.0
72.4
59.1
71.5
73.3
Deepseek Coder
6.7B
23.1
29.3
34.5
26.5
36.3
42.7
General Code
Model
CodeQwenl.55
7B
28.8
38.8
43.3
25.8
29.0
Wwen2.5 Coder Instruct
7B
30.1
49.2
$5.9
33.2
52.5
57.7
RTI.Coder
6.7B
35.8
40.3
43.1
43.5
48.1
CodeV(Qwenl 55)
7B
36.6
53.3
61.3
48.1
56.9
Verilog-Spccific Model
CodeV(Qwen2.5)
39.3
63.5
74.2
41.0
60.1
68.1
Origen
50.6
68.3
74.3
50.9
64.0
VeriSeek
6.7B
29.3
47.1
53.1
31,9
VeriPrefer
7B
53.2
67.7
52.4
66.4
QiMeng-SALY (Ours)
7B
62.6
75.1
81.1
62.0
71.7
76.0
General- purpose Foundation Models: GPT-3.5, GPT-4o, 4PT4 [34], and DeepSeek-v3 13 which demonstrate broad capabilities across diverse domains. 2) General Code Models: CodeQwen1.55 [35, Qwen2.5 Coder Instruct [30] and Deepseek Coder [36], possessing strong coding proficiency but lacking Verilog-specific optimization. 3) Domain-Specialized Verilog Models: Including RTI. Coder
[16], Code (4. Origen [17], VeriSeek 7 and VeriPrefer 28 which employ module-level rewards
in reinforcement learning.
4.2
Main Results
Table coblee@compreeenssiee compare the performance of our QiMeng-SALY method against
baseline models across the VerilogEval [9] [0] and RTLLM [1] benchmarks. To establish a more rigorous evaluation framework, we re-implemented CodeV using Qwen2.5 Coder Instruct as its foundation model, replacing its original CodeQwen 1.5 which demonstrated inferior performance. This architectural alignment with our base model eliminates potential confounding factors arising from fundamental model discrepancies.
Comparison results in Table I and Table show that QiMeng- establishes new lfte-off-hhe-rr results across both benchmarks in the open-source domain. As shown in Table [] in VerilogEval, QiMeng-SALV achieves leading performance among open-source solutions both specification understanding and code completion tasks, attaining 81.4 pass @ on Machine and 60.4 pass @ on Human in VerilogEval 1.0, well as 57.1 (T 0) Specification and 62.2 (T O) Completion in VerilogEval2.00 Its performance is comparable to GPT-4o on the VerilogEval a.. Human benchmark and even surpasses Deepseek v3 and GPT-4o on the VerilogEvall Machine.
As shown in Table 2 QiMeng-SAMe achieves a remarkable 62.6% functional pass @ accuracy on the RTLLM v1.1 benchmark and 62.0% on RTLLM v2.0 with merely 7B parameters, significantly exceeding all existing open-source alternatives and rivaling the performance of DeepSeek-v3, a 671B parameter model. Impressively, its functional pass @ 10 accuracy reaches 81.1 % on RTLLM v1.1, surpassing DeepSeek-v3's 72.4%.
Remarkably, when compared to CodeV (Qwen2.5), trained on identical datasets with the same base model, QiMeng-SALV shows substantial improvements: about 59.7% improvement pass@) accuracy on RTLLM vl.1, and (T=))5500 subtasks. These findings underscore that leveraging RL to learn from correct signal-level imple- mentations yields substantially stronger performance compared to models trained exclusively via supervised fine-tume tuning.
improvement
VerilogEval2.03 specification
Furthermore, when compared with VeriPrefer, which employs module-level rewards in reinforcement learning, our method achieves substantial improvements across all evaluation metrics. This highlights the advantage of fine-grained supervision: signal-level rewards allow the model to effectively learn from partially correct samples that are often disregarded under coarser reward schemes. Consequently, signal-level reinforcement learning enables the generation of more robust and correct Verilog code.

---

(a)
(b)
Figure 3: (a) Pass rate performance comparison between different training stages across different
tasks RTLLM v1.1 benchmark. (b) Signal-aware DPO training datasets scaling RTLLM
benchmark.
4.3 Ablation Study
In this section, present systematic ablation study to evaluate our proposed QiMeng-SALJ Given that RTLLM contains more complex problems than VerilogEval, making it more suitable for revealing nuanced performance differences among models, focus ablation analysis the RTLLM benchmark. Our investigation spans two key dimensions: 1). configuration variations of Signal-aware DPO training, and 2). different training stage implementations of QiMeng-SALV.
Ablation of configuration variations of signal-aware DPO Unlike standard DPO, which limited to processing complete correct modules and must consider all response tokens, our signal signal-aware DPO can not only handle complete correct modules but also extract functional reward in partial correct modules containing correct signal implementations. Therefore, evaluate the efficiency of our method: 1) Complete correct datasets (263k): all of preferred sample in DPO training data are complete correct modules. 2) Partial correct datasets (110k): all of preferred sample DPO training data are partial correct modules containing both correct and incorrect signals. 3) Filter incorrect signals: determines whether erroneous signals in preferred samples participate in DPO computation. As evidenced by the results in Table 3 standard DPO trained exclusively on complete correct datasets achieves superior performance compared training on partially correct datasets. Interestingly, combining both complete and partially correct datasets during training leads to degraded performance relative to using only complete correct datasets. This degradation likely stems from the presence of incorrect signal implementations in partially correct module, which introduce noise and hinder the model's ability discern meaningful reward signals. By contrast, our proposed signal-aware DPO method, which actively filters out incorrect signal implementations, substantially improves performance both scenarios: training on partially correct datasets alone and training on mixed datasets. These findings highlight two key advantages of agnal--wwre DPO: (1) its ability leverage a broader spectrum of training data, and (2) its robustness against noisy signal implementations through effective filtering.
design three different options to
Ablation of different training stage implementations To systematically analyze the performance gains of the model across different training phases, we conducted comprehensive evaluations at each stage. The results, as presented in the Table indicate that SFT yields a substantial improvement of approximately 15 percentage points in all three accuracy dimensions, Subsequent signal-awar DPO training delivers additional gains of 10- 14 percentage points. Figure (a) illustrates the pass rate of the model on the RTLLM v1.1 benchmark tasks across different training stages. The base code model not only answered fewer questions correctly but also exhibited low accuracy for the questions it could answer. After the SFT training stage, the model could correctly answer broader range of questions. Following signal-aware DPO training, the model further improved accuracy on questions could answer while also correctly answering previously unsolvable questions. These findings substantiate that signal-aware DPO effectively utilizes functional correctness feedback signals drive additional performance improvements beyond what is achieved through SFT alone.

---

Table 3: Ablation Different Settings on RTLLM Benchmark
Complete
Partial
Filter
RTLLM vI.1
RTLLM v2.0 (%)
Correct
Correct
Incorrect Signals
Datasets
Datasets
Pass
Pass @5
Pass
Pass@1
Pass@5
Pass@ 10
57.4
74.0
78.3
57.1
70.4
73.4
56.0
67.9
71.6
69.2
55.3
73.1
77.1
55.9
69.8
73.3
56.6
71.0
76.4
54.1
68.4
73.1
62.6
75.1
81.1
62.0
71.7
76.0
Table 4: Ablation Different Training Stage RTLLM Benchmark
RTLLM v1.1 (%)
RTLLMM v2.0 (%)
Model
Pass @
Pass@5
Pass 10
Pass@
Pass @
Pass @ 10
Base Model(Qwen2.5)
30.1
49.2
55.9
33.2
52.5
57.7
+ SFT Signal-aware DPO
48.3
65.2
69.8
50.4
64.3
68.7
62.6
75.1
81.1
62.0
71.7
76.0
To examine the influence of training dataset size on Signal-aware DPO's efficacy, we conducted experiments by training models with preference datasets of different scales. Figure (b) reveals consistent upward trend in pass 1, pass @5, and pass@ metrics as the dataset size grows. These results indicate that incorporating more meaningful functional rewards during reinforcement learning training significantly contributes to improved model performance.
Scaling on Signal-aware DPO training datasets
4.4 Runtime Analysis
We measured the average runtime of simulation and AST parsing over all Verilog samples in training data. average, simulation takes 0.0391 seconds and AST parsing takes 0.0957 seconds per sample. By utilizing 60 CPU cores for parallel processing, we reduce the per-sample average simulation time 0.65 milliseconds and AST parsing time 1.59 milliseconds. These results show that the overhead of simulation and AST parsing is negligible in practice.
5 Conclusion
In this paper, we propose GiMeng-SALL) novel reinforcement learning method for training Verilog code generation, which learns correct signal implementations from erroncous modules, thereby increasing functional correctness rewards and improving capability. Experiments show that our model achieves state-of-the-arr performance VerilogEval and RTLLM, significantly outperforming other open-source models of comparable scale. On RTLLM v1.1, it reaches pass @ 62.6% and pass @5 75.1%, matching the performance of DeepSeek-v3 v3 (671B parameters) with only 7B parameters. QiMeng-SALY shifts the training paradigm for Verilog code generation from the module level to the signal level.
10

---

References
[1] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization 7074776777 2017.
algorithms," arXiv preprint
[2] Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al., "Deepseekmath: Pushing the limits of mathematical reasoning in open language models," arXiv preprint arXiv:2402.03300 2024.
[3] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, "Direct preference optimization: Your language model is secretly a reward model," Advances in Neural Information Processing Systems, vol. 36, pp. 53728-53741, 2023.
[4] Y. Zhao, D. Huang, C. Li, Jin, Z. Nan, T. Ma. L. Qi. Y. Pan, Z. Zhang, Zhang, al., "Codev: Empowering Ilms for verilog generation through multi-level summarization," arXiv preprint 1xXiv:2407.10424 2024.
[5] S. Thakur, B. Ahmad, H. Pearce, B. Tan, B. Dolan-Gavitt, R. Karri, and S. Garg, "Verigen: A large language model for verilog code generation," " ACM Transactions on Design Automation of Electronic Systems, vol. 29, no. 3, pp. 1-31, 2024.
[6] H. Pearce, B. Tan, and R. Karri, "Dave: Deriving automatically verilog from english," in Proceedings of the 2020 ACM/IEEE Workshop on Machine Learning for CAD, pp. 27-32, 2020.
[7] N. Wang, B. Yao. J. Zhou, X. Wang, Z. Jiang, and N. Guan, "Large language model for verilog generation with golden code feedback," arXiv preprint arXiv:2407.18271 2024.
[8] P. Wu, N. Guo, X. Xiao, W. Li, X. Ye, and D). Fan, "Itertl: An iterative framework for fine-tuning Ilms for rtl code generation," arXiv preprint arXiv:2407. 12022, 2024.
[9] M. Liu, N. Pinckney, B. Khailany, and H. Ren, "Verilogeval: Evaluating large language models for verilog code generation." " in 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD), pp. 1-8, IEEE, 2023.
[10] N. Pinckney, C. Batten, M. Liu, H. Ren, and B. Khailany, "Revisiting verilogeval: Newer Ilms, in-context learning, and specification-to-rll tasks," arXiv preprint arXiv:2408. 1053, 2024.
[11] Y. Lu, S. Liu, Q. Zhang, and Z. Xie, "Rtllm: An open-source benchmark for design rtl generation with large language model," 2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC), pp. 722-727 IEEE, 2024.
[12] Liu, Lu, Fang, M. Li, and Z. Xie, "Openllm-rtl: Open dataset and benchmark for Ilm- aided design rtl generation," in Proceedings of the 43rd IEEF/ACM International Conference on Computer-Aioe Design, pp. 2024.
[13] A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al., "Deepseek-ve technical report," arXiv preprint arXiv:2412.19437 2024.
[14] Z. Pei, H. Zhen, M. Yuan, Y. Huang, and B. Yu, "Betterv: Controlled verilog generation with discriminative guidance," in ICML. OpenReview.net, 2024.
[15] M. Gao, J. Zhao, Z. Lin, W. Ding, X. Hou, Y. Feng, C. Li, and M. Guo, "Autovcoder: A systematic framework for automated verilog code generation using Ilms," in 2024 IEEE 42nd International Conference on Computer Design (ICCD), pp. 162-169, IEEF, 2024.
[16] S. Liu, W. Fang. Y. Lu, Q Zhang, H. Zhang, and Z. Xie, "Rtlcoder: Outperforming gpt-3.5 in design rtl generation with our open-source dataset and lightweight solution," " in 2024 JEEE LLM Aided Design Workshop (LAD), pp. 1-5, IEEE, 2024.
[17] Cui, C. Yin, K. Zhou, Y. Xiao, Sun, Q. Xu, Q. Guo, Y. Liang, X. Zhang, D. Song, el al., "Origen: Enhancing rtl code generation with code-to-code augmentation and self-reflection," in Proceedings of the 43rd IEFE/ACM International Conference on Computer-Aide Design
pp. 2024.
11

---

[18] G. Tie, Z. Zhao, D. Song, F. Wei, R. Zhou, Y. Dai, W. Yin, Z. Yang, J. Yan, Y. Su, et al., "A survey on post-training large language models," arXiv preprint arXiv:2503.06072 2025.
[19] Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al., "Training a helpful and harmless assistant with reinforcement learning from human feedback," arXiv preprint arXiv:2204.05800 2224400662, 2022.
[20] H. Dong, W. Xiong, D. Goyal, Y. Zhang, W. Chow, R. Pan, S. Diao, J. Zhang. K. Shum, and T. Zhang, "Raft: Reward ranked finetuning for generative foundation model alignment," arXiv preprint 2xX1v:2304.06767 2023.
[21] H. Yuan, Z. Yuan, C. Tan, W. Wang, S. Huang, and F. Huang, "Rrhf: Rank responses to align language models with human feedback," Advances in Neural Information Processing Systems, vol. 36, pp. 10935- 10950, 2023.
[22] Y. Zhao, R. Joshi, T. Liu, M. Khalman, M. Salch, and P. Liu, "Slic-hf: Sequence likelihood calibration with human feedback," " arXiv preprint arXiv:2305. 10425, 2023.
[23] T. Liu, Y. Zhao, R. Joshi, M. Khalman, M. Salch, P. J. Liu, and J. Liu, "Statistical rejection sampling improves preference optimization," arXiv preprint arXiv:2309.06633 2023.
[24] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al., "Evaluating large language models trained on code," arXiv preprint arXiv:2107.03374 2021.
[25] H. Le, Y. Wang, A. Gotmare, S. Savarese, and S. C. H. Hoi, "Coderl: Mastering code generation through pretrained models and deep reinforcement learning," Advances in Neural Information Processing Systems, vol. 35, pp. 21314-21328, 2022.
[26] J. Liu, Y. Zhu, K. Xiao, Q. Fu, X. Han, W. Yang, and D. Ye, "Ritf: Reinforcement learning from unit test feedback," arXiv preprint w 2023.
[27] S. Dou, Y. Liu, H. Jia, L. Xiong, E. Zhou, W. Shen, J. Shan, C. Huang, X. Wangg X. Fan, et al., "Stepcoder: Improve code generation with reinforcement learning from compiler feedback," arXiv preprint wxXiv:2402.01399 2024.
[28] N. Wang, B. Yao, Zhou, Y. Hu, Wang, Guan, and 7. Jiang, "Insights from verification: Training a verilog generation Ilm with reinforcement learning with testbench feedback," arXiv preprint arXiv:2504. 15804, 2025.
[29] C. Wolf, J. Glaser, and J. Kepler, "Yosys-a free verilog synthesis suite," " in Proceedings of the 21 Austrian Workshop Microelectronics (Austrochip), vol. 97, 2013.
[30] B. J.i, Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu, el al., "Qwen2. 5-coder technical report," arXiv preprint 120Xii:2409.121880 2024.
[31] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, et al., "Lora: Low-rank adaptation of large language models.." ICLR, vol. no. p. 3, 2022.
[32] D. P. Kingma and Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6900 2014.
[33] A. Holtzman, J. Buys, L. Du M. Forbes, and Y. Choi, "The curious case of neural text degeneration," arXiv preprint 1rXivv/904.09955 2019.
[34] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al., "Gpt-4 technical report," arXiv preprint arXiv:2303.08774.200 2023.
[35] J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng. Y. Fan, W. Ge, Y. Han, F. Huang, B. Hui, L. Ji, M. Li, Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, Ren, C. Tan, S. Tan, J. Tu, P. Wang. S. Wang. W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang. J. Yang, S. Yang, Y. Yao, B. Yu, H. Yuan, Z. Yuan, J. Zhang, X. Zhang, Y. Zhang, Z. Zhang, C. Zhou,
J. Zhou, X. Zhou, and T. Zhu, "Qwen technical report," arXiv preprint arXiv:2309. 16609, 2023.
12

---

[36] D. Guo. Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y. Wu, Y. Li, et al., "Deepseek-coder: When the large language model meets programming the the rise of code intelli- gence," arXiv preprint arXiv:2401. 14196, 2024.
13

---

Supplementary Material
A Experiment Compute resources
For the SFT phase, execute full-pan full-parameter optimization across 2 training epochs utilizing cluster of NVIDIA A100-80GB SMX GPUs, with the complete training procedure consuming approximately 20 hours. Subsequently, during the Signal-aware DPO stage, implement LoRA- based fine-tuning over 7000 steps on an array of NVIDIA A100-40GB GPUs, resulting in a total training duration of roughly 15 hours.
B
Societal Impacts
Our research delivers significant positive societal impact by substantially improving the functional correctness automatically generated Verilog code. This advancement enhances productivity in circuit design workflows, accelerates development cycles, and provides industry practitioners with reliable assistive tool. However, we acknowledge potential negative implications in academic settings. The model's 's capabilities could be misused by students to complete Verilog programming assignments or examinations, potentially facilitating academic dishonesty. We emphasize the importance of developing appropriate usage guidelines and detection mechanisms mitigate such risks while preserving the technology's 's beneficial applications.
C Limitation
QiMeng-SALV determines the functional correctness of the generated module by producing random input signals and comparing the output signals between the generated module and the reference module. If the reference module in the training dataset itself is incorrect, may lead erroneous judgments of functional correctness, necessitating a relatively high-quality training dataset. In this paper, we do discuss how obtain high-quality dataset, as this is beyond our scope. In our experiment, we use the CodeV dataset, a high gigh-qualtty dataset obtained by crawling all publicly available Verilog module code on GitHub. We also perform data cleaning ensure the reference modules are syntactically correct and compilable.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
In Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes)
Justification:
We will open-source our model huggingface.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
We will open-source our datasets and code on github.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
httppsscccpblbccpppeeccccc guwdes/ddeeeeee Cooeeeoodeesssseeeeee http///bbb////uuuuoooooo//aaaaaa casssiinnccccyyc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: In
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Our experiments rely heavily large language model (LLM) training, and repeating each setting multiple times would significantly increase the cost.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research strictly adheres to the NeurIPS Code of Ethics. All experiments were conducted with integrity, transparency, and respect for human and environmental welfare.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: In Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the original paper.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: This research does not involve LLMs any important, original, non- standard components.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
21


Reviews:
[{"review_id": "KIG0Z8VX5Q", "reviewer": "Reviewer_q7XC", "summary": "A key challenge in RL-based code generation for hardware is the lack of good rewards, existing approaches rely on structural similarity between generated and reference code, which doesn’t always reflect functional correctness. SAL-V tackles this by shifting the optimization focus from the entire module level to the level of individual output signals. they extract these correct signal segments by checking signal correctness via simulation and identifying the corresponding AST subtrees. Their approach leads to stronger and more targeted learning signals during RL", "questions": "Can you report training cost and overhead introduced by simulation + AST parsing?\nCould you compare signal-wise rewards to other fine-grained alternatives?\nIs SAL-V effective when used with other RL algorithms beyond DPO?\nCan you include qualitative examples and error analysis of signal-wise segment extraction?\nHow does SAL-V perform when signal outputs are interdependent?\nHow reliable is AST extraction when the generated code is invalid or malformed?", "limitations": "yes in appendix.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Signal-level reward formulation is novel and well-justified,  shifts RL optimization from module-level to signal-level using functionally verifiable outputs, enabling learning from partially correct generations.\n- Seamless integration into Direct Preference Optimization training. The method doesn’t require architecture changes, making it easy to plug into existing SFT+DPO pipelines for code generation.\n- Assumption of signal independence is untested, no experiments probe scenarios with dependent or correlated signals, which are common in real designs, is it a safe assumption? All evaluations are conducted on curated, synthetic datasets.\n- No runtime analysis is presented to assess overhead of simulation and AST parsing.\n- Comparison with alternative reward schemes?\n\nThe paper presents a creative, well-motivated, and technically sound method for improving reward fidelity in Verilog code generation. However, the paper would benefit from more thorough analysis in several areas—robustness to signal dependencies, runtime costs, qualitative behavior, and evaluation beyond curated benchmarks. These limitations suggest that while the core contribution is strong, its generalization and practical deployment may still require further work.", "was_revised": true, "final_justification": "I am maintaining my score of 4 (Borderline Accept). The core contribution of formulating RL rewards at the signal level in Verilog code generation is both novel and technically well-grounded. The authors show that the signal independence assumption is valid empirically which makes a strong case for the paper.", "rebuttal": "**Comments on Weaknesses:**\n\n**W1: Assumption of signal independence is untested, no experiments probe scenarios with dependent or correlated signals, which are common in real designs, is it a safe assumption? All evaluations are conducted on curated, synthetic datasets.**\n\nThank you for the thoughtful feedback. We carefully examined the assumption both conceptually and empirically and our analysis shows that the assumption of signal independence is safe.\n\nIn our setup, we consider an output signal to be independent of another if its minimal and complete implementation does not rely on the other output signal. Even if two output signals depend on a shared intermediate signal, as long as one does not directly or indirectly consume the value of the other, we treat them as independent. This assumption allows us to isolate and reward correct output signal code segments without being affected by incorrect ones.\n\nTo evaluate the safety of this assumption in practice, we conducted a large-scale analysis using AST parsing on all training samples that contain both correct and incorrect output signals. We found that only 4.8% (1,760 out of about 36k) of these samples exhibited cases where a correct signal directly or indirectly depended on an incorrect one. Given this low occurrence, we filtered out such samples during training.\n\nThus, both by design and through empirical analysis, we conclude that the signal independence assumption is reasonable and does not compromise the validity or effectiveness of our approach. We emphasize that signal-aware code extraction is applied only during training to improve learning. During inference, we apply no additional processing and use the same benchmarks as prior work.\n\n**W2: No runtime analysis is presented to assess overhead of simulation and AST parsing.**\n\nThank you for raising this point. Our experiment shows that the overhead introduced by simulation and AST parsing is negligible.\nWe measured the average runtime of simulation and AST parsing over all Verilog samples in training data. On average, simulation takes 0.0391 seconds and AST parsing takes 0.0957 seconds per sample. By utilizing 60 CPU cores for parallel processing, we reduce the per-sample average simulation time to 0.65 milliseconds and AST parsing time to 1.59 milliseconds. These results show that the overhead of simulation and AST parsing is negligible in practice.\n\n**W3: Comparison with alternative reward schemes?**\n\nThank you for this insightful feedback. To the best of our knowledge, SAL-V is the first method to introduce a fine-grained, signal-level reward mechanism for Verilog code generation. Thus, we conduct a thorough comparison with the most relevant alternative: standard module-level reward used in conventional DPO on RTLLM v1.1. The result is also presented in Table 3 of the ablation study.\n| Reward Type | ｜  | |Syntax(%) | |｜| | Func (%) | |\n|:--:|:--:|:-:|:--:|:-:|:--:|:-:|:-:|:-:|\n|  | ｜ | Pass@1 | Pass@5 | Pass@10 | ｜|Pass@1 | Pass@5 | Pass@10 |\n| Module-level |｜ | 84.6   | 90.1   | 92.1 | ｜|52.9| 70.6 | 76.2 |\n| Signal-level  | ｜| **91.8** | **96.2** | **96.5**    | ｜| **61.5**| **75.9** | **80.3**|\n\nAs the results demonstrate, the signal-level reward method outperforms the module-level approach across all evaluation metrics. This indicates that signal-level rewards offer more fine-grained supervision, allowing the model to learn from partially correct samples and ultimately improving overall performance when such samples are included.\n\n**Reply to Questions:**\n\n**Q1: Can you report training cost and overhead introduced by simulation + AST parsing?**\n\nThank you for the thoughtful question. For the overhead of simulation and AST parsing, please refer to W2. We would like to clarify that both simulation and AST parsing are performed offline as part of the dataset preprocessing pipeline and do not incur training-time overhead.\nMoreover, modifying the DPO loss for signal-aware reward computation does not introduce additional training cost.\n\n**Q2: Could you compare signal-wise rewards to other fine-grained alternatives?**\n\nThank you for the thoughtful question. Please refer to W3.\n\n**Q3: Is SAL-V effective when used with other RL algorithms beyond DPO?**\n\nThank you for the excellent question. To assess the scalability and general applicability of SAL-V, we implemented it in an online RL setting using GRPO [1], an improved version of PPO [2]. The results are summarized in the following table:\n\n| Reward Type |  Model |｜ |   |   RTLLM   |  | ｜|   |   Machine  | |｜|   |  Human   | |\n|:---:|:---:|:---:|:---:|:---:|:---:|---:|:---:|:---:|:---:|-----:|:----:|:-----:|:---:|\n|   |   | ｜ | Pass@1 | Pass@5 | Pass@10 | ｜ | Pass@1 | Pass@5 | Pass@10 | ｜| Pass@1 | Pass@5 | Pass@10 |\n| Signal-level | SAL-V DPO   |  ｜| 61.5  |  **75.9**  |  **80.3**   |  ｜ |   79.8 |  88.0  |  **90.3**   |  ｜|  60.1 |  67.6  |**70.4** |\n| Module-level | Normal GRPO |  ｜|57.7  |  72.6  |  78.4   |  ｜|  82.6  |  86.0  |  86.8   |｜|  60.9 |  65.2  |66.5|\n| Signal-level | SAL-V GRPO  |  ｜ |    **61.5**  |  75.1  |  80.1   |  ｜| **84.7**  |  **88.2**  |  88.9   |｜ | **64.5**   |  **68.4**  |69.3|\n\n\nAs shown, online SAL-V (GRPO) outperforms offline SAL-V (DPO) on both VerilogEval-Machine and VerilogEval-Human, while achieving comparable performance on RTLLM. This demonstrates that SAL-V is not only effective in offline settings (e.g., DPO), but also generalizes well to online reinforcement learning frameworks like GRPO.\n\nMoreover, signal-level SAL-V with GRPO consistently outperforms its module-level counterpart across all benchmarks. These results further highlight the benefit of using fine-grained and informative reward signals, which help the model learn more effectively and generalize better across tasks.\n\n[1] Shao Z, Wang P, Zhu Q, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models[J]. arXiv preprint arXiv:2402.03300, 2024.\n\n[2] Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms[J]. arXiv preprint \n\n**Q4: Can you include qualitative examples and error analysis of signal-wise segment extraction?** \n\nYes, for example, consider a training sample involving the design of a controller for a 4-digit 7-segment display. The model generates the following code:\n\n```\nmodule anode_controller(\n    input [1:0] refresh_count,\n    output reg [3:0] anode,\n    output reg dp\n    );\n    always @ (*)\n        case(refresh_count)\n        2'b00: begin \n          anode = 4'b1110; \n          dp = 0; \n        end\n        2'b01: begin \n          anode = 4'b1101; \n          dp = 0; \n        end\n        2'b10: begin \n          anode = 4'b1011; \n          dp = 1; \n        end\n        2'b11: begin \n          anode = 4'b0111; \n          dp = 1; \n        end\n        default: anode = 4'b1111;\n        endcase   \nendmodule\n```\nIn this case, the output signal anode is correct, while dp is incorrect. In SAL-V, when we extract the code relevant to the anode signal for reward assignment, the extracted minimal code snippet is:\n```\nmodule anode_controller(\n    input [1:0] refresh_count,\n    output reg [3:0] anode,\n    );\n    always @ (*)\n        case(refresh_count)\n        2'b00: begin \n          anode = 4'b1110; \n        end\n        2'b01: begin \n          anode = 4'b1101; \n        end\n        2'b10: begin \n          anode = 4'b1011; \n        end\n        2'b11: begin \n          anode = 4'b0111; \n        end\n        default: anode = 4'b1111;\n        endcase\nendmodule\n```\nThis extracted code snippet contains only the minimal set of tokens required to generate the correct anode signal and doesn't contain any incorrect output signal. During signal-aware DPO training, only the tokens corresponding to this relevant subset are included in the loss computation, allowing the model to focus learning on the functionally correct components.\n\n**Q5: How does SAL-V perform when signal outputs are interdependent?** \n\nThank you for the thoughtful question. As discussed in W1, we treat output signals as independent if their minimal and complete implementations do not directly rely on each other. Even if two signals share intermediate signal computations, they are considered independent as long as one does not consume the other’s value. This allows us to isolate and reward correct output signal code segments without being affected by incorrect ones.\n\nIn cases where an incorrect signal depends on a correct one, SAL-V still performs well, since it isolates and rewards only the code segments related to correct signals. However, when a correct signal depends on an incorrect one, the reward becomes unreliable and such samples are not suitable for effective RL training. As shown in our W1 analysis, such cases are rare, only 4.8% of multi-signal samples, so we filter them out during training to ensure stability.\n\n**Q6: How reliable is AST extraction when the generated code is invalid or malformed?**\n\nThank you for your valuable feedback. In fact, our AST extraction is applied only to syntactically valid Verilog code, all invalid or malformed code samples are filtered out during the construction of the signal-aware dataset. This ensures that the input to our signal extraction pipeline is always compilable, allowing reliable AST construction and rule-based analysis of signal-relevant code segments.\n\nTo further evaluate the robustness of our method, we conducted a compilation-based validation experiment on all generated samples in RL training stage containing at least one correct signal. We extract the related code segment for correct signals in each sample and test their compilability using iverilog [1]. We observed 94.9% compile success rate among these extracted code, indicating that these extracted segments are both syntactically valid and compilable. These results confirm the reliability and scalability of our AST-based signal extraction methodology.\n\n[1] S. Williams, “Icarus verilog,” https://github.com/steveicarus/iverilog, 2000."}, {"review_id": "rLXZVW2sYm", "reviewer": "Reviewer_voDh", "summary": "This paper introduces SAL-V, a reinforcement learning (RL) approach to Verilog code generation, emphasizing fine-grained signal-aware rewards and Direct Preference Optimization (DPO). Unlike conventional methods that use module-level rewards, SAL-V employs signal-level correctness rewards extracted through Yosys AST parsing and dependency analysis, guiding the generation of Verilog code. Experimental evaluation demonstrates significant performance improvements on established benchmarks, surpassing several strong baselines.", "questions": "1.\tCould you clarify which base models were used to train SAL-V and how they were configured (e.g., frozen layers, RL steps, DPO parameters)?\n2.\tWhy were simpler RL strategies like module-level reward (e.g., from Insights from Verification) not included for comparison?\n3.\tHow does your signal extraction pipeline handle ambiguous or non-standard Verilog code?\n4.\tIs the Yosys-based extraction scalable to large, complex modules?\n5.\tCould you provide a brief overview of the distribution of your signal-aware dataset—how many signals, what kinds, and how are preference pairs balanced?", "limitations": "The authors partially acknowledge limitations, but further elaboration is necessary on failure modes of signal-level extraction, scalability, and comparisons with simpler methods.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The proposed signal-level correctness reward approach significantly refines traditional module-level RL feedback, potentially offering better learning signals.\n2. Evaluations across VerilogEval2.0 and RTLLM benchmarks demonstrate the method’s superior performance compared to strong baselines such as GPT-4o and DeepSeek-v3.\n3. DPO provides more stable training compared to PPO, aligning effectively with the structured and fine-grained nature of Verilog code.\n4. Detailed ablation studies demonstrate the contributions of various components of the proposed method.\n\nWeaknesses:\n1. The baseline models' details, such as their specific architectures and training setups, are not clearly articulated. The paper does not explicitly state or adequately discuss the choice of base models.\n2. Presentation Issues: Some typographical issues were observed, including inconsistent decimal alignment in Table 1 for \"CodeV(Qwen1.5)\" and suboptimal sizing of Figure 2(a), making the framework overview challenging to interpret clearly.\n3. Incomplete Related Work Discussion: While several recent works have explored RL-based RTL generation, such as \"Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback,\" the authors do not provide comparative analysis with these related approaches. Given the lack of theoretical justification and limited experimental settings, omitting such comparisons makes the conclusions regarding the superiority and necessity of the signal-aware method less convincing.", "was_revised": true, "final_justification": "The authors have addressed my concerns and I would like to raise my score from 3 to 4.", "rebuttal": "**Comments on Weaknesses:**\n\n**W1: The baseline models' details, such as their specific architectures and training setups, are not clearly articulated. The paper does not explicitly state or adequately discuss the choice of base models.**\n\nThank you for your valuable feedback. We clarify the choice of base model and the associated training setup as follows:\n\nWe used Qwen2.5 Coder Instruct as the base model, as our goal is to train a code generation model (not a code completion model like CodeV) that can generate full Verilog modules, even without module headers in the prompt. Qwen2.5 Coder Instruct is well-suited for generating structured programming code and easily transferable to Verilog generation tasks.\n\nWe first fine-tuned Qwen2.5 Coder Instruct on a 162k cleaned CodeV dataset to obtain an SFT model. During the SFT stage, we performed full fine-tuning for 2 epochs with a learning rate of 1e-5 and batch size of 64.\n\nUsing the SFT model, we sampled 5 outputs for each Verilog design and collected 143k preference pairs based on signal correctness: generations with correct signals were treated as preferred samples, and incorrect ones as dispreferred. We then applied our signal-aware DPO method to fine-tune the SFT model using LoRA. Specifically, we applied LoRA to the linear projection matrices ($W_q, W_k, W_v, W_{out}$) of each transformer layer. The DPO fine-tuning used a batch size of 64, a learning rate of 5e-6, and was trained for 3 epochs (6,756 steps in total).\n\n**W2: Presentation Issues: Some typographical issues were observed.**\n\nThank you for pointing out these presentation issues. We will fix these issues in the revised paper.\n\n**W3: Incomplete Related Work Discussion: While several recent works have explored RL-based RTL generation, such as \"Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback,\" the authors do not provide comparative analysis with these related approaches.**\n\nThank you for your insightful feedback. While VeriPrefer also leverages functional rewards for training, our method differs in several key aspects:\n\n1. Reward Granularity: VeriPrefer uses module-level rewards, meaning only entirely correct modules receive a reward. In contrast, our method introduces signal-level rewards, a more fine-grained and novel approach. This allows partially correct modules to still contribute useful feedback during training if they contain correctly implemented signals. As a result, our method provides denser and more informative rewards, which leads to better RL performance.\n\n2. Correctness Evaluation: Our approach evaluates signal correctness by executing both the testbench and golden reference code with randomly generated inputs and comparing their output signals. VeriPrefer, on the other hand, relies on general-purpose testbench generation without access to reference code, using commercial LLMs. This may lead to unreliable or noisy feedback during training.\n\n3. Training Paradigm: We propose a signal-aware DPO method specifically tailored for Verilog code generation, enabling fine-grained RL training at the signal level. VeriPrefer applies a standard DPO pipeline without such signal-level supervision.\n\nThese distinctions enable our method to provide a denser and more reliable reward signal, making it better suited for improving model performance in the context of Verilog code generation. We will incorporate this comparative analysis into the revised version of the paper.\n\n**Reply to Questions:**\n\n**Q1: Could you clarify which base models were used to train SAL-V and how they were configured (e.g., frozen layers, RL steps, DPO parameters)?**\n\nThank you for the thoughtful question. Please refer to W1.\n\n**Q2: Why were simpler RL strategies like module-level reward (e.g., from Insights from Verification) not included for comparison?**\n\nThank you for this insightful feedback. Our ablation study actually includes a comparison between module-level and signal-level rewards on RTLLM v1.1 in Table3 in our paper. \n|Reward Type|｜||Syntax(%) | |｜| | Func (%)||\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n||｜|Pass@1| Pass@5 | Pass@10 | ｜|Pass@1 | Pass@5 |Pass@10|\n|Module-level|｜|84.6|90.1|92.1|｜|52.9|70.6|76.2|\n|Signal-level|｜|**91.8**| **96.2**| **96.5**|｜|**61.5**|**75.9**|**80.3**|\n\nWe also provide the comparison between SAL-V and VeriPrefer on RTLLM v1.1, VerilogEval Machine and Human benchmarks:\n\n|Reward Type|Model|｜|RTLLM| |｜| Machine | |｜| Human| |\n|-|-|-|-|-|-|-|-|-|-|-|\n| | |｜|pass@1|pass@5|｜|pass@1|pass@5|｜|pass@1|pass@5|\n|Module-level| VeriPrefer |｜| 53.2   |67.7|｜|72.7|85.8|｜|49.7|62.3|\n|Signal-level|SAL-V|｜|**61.5**|**75.9**|｜|**79.8**|**88.0**|｜|**60.1**|**67.6**|\n\nAs the results show, the signal-level reward method consistently outperforms the module-level approach across all evaluation metrics. This demonstrates the advantage of fine-grained supervision: signal-level rewards enable the model to effectively learn from partially correct samples, which are often ignored in coarser reward schemes. As a result, signal-level RL leads to more robust and performant Verilog code generation.\n\n**Q3: How does your signal extraction pipeline handle ambiguous or non-standard Verilog code?**\n\nThank you for your valuable feedback. In SAL-V, our signal extraction pipeline is designed to be robust by operating only on code samples that produce valid execution results. This ensures that the code is syntactically correct and compilable, allowing us to reliably construct ASTs and extract signal-relevant code segments through a rule-based analysis. While ambiguous or non-standard Verilog code may occur in model outputs, such samples typically fail during simulation or compilation and are therefore excluded from the signal extraction pipeline.\n\nTo assess the reliability of our extraction process, we conducted a compilation-based evaluation on all the generated samples from the signal-aware DPO stage that contain at least one correct signal. After extracting the relevant code segments of correct signals, we tested their compilability using iverilog [1]. The compilation success rate was 94.9%, indicating that the extracted code is not only minimal and complete, but also syntactically valid and compilable.\nThese results confirm the robustness and reliability of our AST-based signal extraction pipeline.\n\n[1] S. Williams, “Icarus verilog,” https://github.com/steveicarus/iverilog, 2000.\n\n**Q4: Is the Yosys-based extraction scalable to large, complex modules?**\n\nYes, our Yosys-based signal extraction method is scalable to large and complex Verilog modules. Such modules are typically composed of hierarchically nested submodules. Our AST-based analysis supports recursive traversal across module instantiations, allowing signal dependencies to be accurately traced through multiple layers of the design hierarchy.\nLet us illustrate this with an example:\n```\nmodule lab1_3 (a, b, c, aluctr, d, e);\n   input [1:0] aluctr;\n   input [3:0] a, b, c; \n    output [3:0] d;\n    output [3:0] e;\n    wire [2:0] sel;\n    assign sel = {aluctr[1], aluctr[0], aluctr[0]};\n    mux7 MA (.a(d), .b(e), .c(c), .sel(sel), .out(e[0]));\n    mux7 MB (.a(b), .b(~a), .c(a), .sel(sel), .out(e[1]));\n    mux7 MC (.a(b), .b(a), .c(~a), .sel(sel), .out(e[2]));\n    mux7 MD (.a(a), .b(b), .c(a^b), .sel(sel), .out(e[3]));\n    ALU adder0(a,b,c,aluctr,d);\nendmodule  \nmodule mux7(a, b, c, sel, out);\n   input [3:0] a, b, c;\n   input [2:0] sel;\n    output out;\n   assign out = sel[2] ? (sel[1] ? (sel[0] ? c[3] : c[2]) : (sel[0] ? c[1] : c[0])) :\n                       (sel[1] ? (sel[0] ? b[3] : b[2]) : (sel[0] ? b[1] : b[0]));\nendmodule\nmodule ALU(a, b, cin, aluc, sum); \n....\nendmodule\n```\nSuppose we aim to extract the code relevant to signal `d` in the top-level module `lab1_3`. Through AST analysis, we determine that signal `d` is connected to the signal `out` of a submodule `mux7`, which is instantiated inside the top module. Within `mux7`, `out` depends on signals `a`, `b`, `c`, and `sel`. These internal signals correspond to `a`, `b`, `c`, and `aluctr` in module `lab1_3`. As a result, we determine that `d` depends only on `a`, `b`, `c`, and `aluctr`, and is independent of other signals such as `sel` and `e` in module `lab1_3`. \n\nThis recursive dependency tracing enables us to extract only the minimal and functionally complete subset of code relevant to signal `d`, even in the presence of deep modular hierarchies. Thus, the methodology remains robust and scalable for large and complex designs.\n\n**Q5: Could you provide a brief overview of the distribution of your signal-aware dataset—how many signals, what kinds, and how are preference pairs balanced?**\n\nYes, thank you for your valuable feedback. Below is a brief overview of our signal-aware dataset and the construction of preference pairs. We will also include these details in the revised version of the paper for clarity and completeness.\n\nAmong the 165k Verilog design problems in CodeV, 124k are simulation-valid. Of these, 71k (57.8%) are single-output signal tasks, while 49k (39.2%) involve multiple output signals, highlighting the prevalence of multi-signal designs.\n\nFor each problem, we sampled 5 generations and constructed preference pairs by treating generations containing at least one correct signal as preferred, and those without any correct signals as dispreferred. After removing duplicates, we obtained 143k preference pairs for training our signal-aware DPO model.\n\nAmong them, 106k pairs have preferred samples that are entirely correct (no incorrect signals). We denote this as the Complete Correct Dataset. The remaining 43k are Partial Correct Dataset pairs, where the preferred sample contains both correct and incorrect signals.\nOur ablation study in section 4.3 further shows that the Partial Correct Dataset contributes valuable reward signals under our signal-aware training framework, enhancing model performance."}, {"review_id": "iXWFNDzkbf", "reviewer": "Reviewer_2wdY", "summary": "This paper proposes SAL-V, a novel signal-aware learning approach for Verilog code generation. The key idea is to leverage functionally correct output signal code segments to optimize RL training. The method addresses the challenge of insufficient functional rewards in LLM-based Verilog generation by identifying and utilizing correct signal-level implementations from partially incorrect modules. It introduces a signal-aware DPO that focuses on correct signal-level code segments. The approach achieves state-of-the-art performance, demonstrating a significant advancement in the field.", "questions": "Please refer to the weaknesses.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper introduces a new paradigm for Verilog code generation, shifting from module-level to fine-grained signal-level optimization. This is a crucial innovation that addresses a fundamental limitation of existing RL-based methods, which struggle with sparse functional rewards.\n\n\n2. SAL-V's ability to extract meaningful functional rewards from erroneous modules by verifying and isolating correct signal implementations is highly impactful. The use of AST for signal-aware code extraction is a clever technical contribution.\n\n\n3. The experimental results are compelling, showing that SAL-V achieves competitive and even superior performance compared to much larger models (e.g., matching DeepSeek-V3 671B with a 7B model) on challenging benchmarks like VerilogEval2.0 and RTLLM.\n\n\nWeaknesses:\n1. While the proposed signal-aware learning is highly effective, it would be beneficial to see the methodology validated on large-scale on-policy RL. The potential of off-policy RL for very large-scale training might be limited, and demonstrating the applicability and scalability of SAL-V within an on-policy framework would further strengthen the paper's claims and implications for future research.", "was_revised": true, "final_justification": "I have read the rebuttal and will maintain my score.", "rebuttal": "**Comments on Weaknesses:**\n\n**W1: While the proposed signal-aware learning is highly effective, it would be beneficial to see the methodology validated on large-scale on-policy RL. The potential of off-policy RL for very large-scale training might be limited, and demonstrating the applicability and scalability of SAL-V within an on-policy framework would further strengthen the paper's claims and implications for future research.**\n\nWe appreciate the reviewer’s suggestion. We initially chose DPO for its sampling efficiency and low computational cost and it allows us to pre-sample generated code before training, making it well-suited for resource-efficient RL.\n\nTo evaluate the scalability of our approach in an online RL setting, we implemented SAL-V using GRPO [1], an improved online RL algorithm built upon PPO [2].\n\n| Reward Type |  Model |｜ |   |   RTLLM   |  | ｜|   |   Machine  | |｜|   |  Human   | |\n|:---:|:---:|:---:|:---:|:---:|:---:|---:|:---:|:---:|:---:|-----:|:----:|:-----:|:---:|\n|   |   | ｜ | Pass@1 | Pass@5 | Pass@10 | ｜ | Pass@1 | Pass@5 | Pass@10 | ｜| Pass@1 | Pass@5 | Pass@10 |\n| Signal-level | SAL-V DPO   |  ｜| 61.5  |  **75.9**  |  **80.3**   |  ｜ |   79.8 |  88.0  |  **90.3**   |  ｜|  60.1 |  67.6  |**70.4** |\n| Module-level | Normal GRPO |  ｜|57.7  |  72.6  |  78.4   |  ｜|  82.6  |  86.0  |  86.8   |｜|  60.9 |  65.2  |66.5|\n| Signal-level | SAL-V GRPO  |  ｜ |    **61.5**  |  75.1  |  80.1   |  ｜| **84.7**  |  **88.2**  |  88.9   |｜ | **64.5**   |  **68.4**  |69.3|\n\nAs illustrated by the results, SAL-V combined with GRPO (online RL) achieves better performance than with DPO (offline RL) on both VerilogEval-Machine and VerilogEval-Human, and performs comparably on RTLLM. This indicates that SAL-V is compatible with both offline and online RL algorithms, showcasing strong generalizability across training paradigms.\n\nIn addition, using signal-level rewards with GRPO consistently outperforms module-level reward schemes on all benchmarks. This highlights the benefit of providing more detailed and targeted feedback during training, allowing the model to better leverage partially correct outputs and improve overall Verilog code generation performance.\n\n[1] Shao Z, Wang P, Zhu Q, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models[J]. arXiv preprint arXiv:2402.03300, 2024.\n\n[2] Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms[J]. arXiv preprint"}, {"review_id": "5T4D69toDM", "reviewer": "Reviewer_EgJ3", "summary": "This paper proposes a signal-aware learning framework (SAL-V) for Verilog code generation, using AST-based extraction of functionally correct code segments to provide fine-grained RL rewards. The approach addresses limitations of module-level rewards and achieves state-of-the-art results on multiple benchmarks, showing strong performance even with smaller models.", "questions": "For Table 1, why does the VeriSeek method achieve such a notably low score on the Specification task?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n- Employing abstract syntax tree (AST) analysis to identify signal-aware code segments is a novel approach for data extraction, and it is well-suited to Verilog code generation. This represents an interesting problem and a promising direction for further exploration.\n- The experimental results validate the effectiveness of the proposed method across multiple datasets, demonstrating its robustness and broad applicability.\n\nWeakness:\n\n- A major weakness is the risk of data contamination in the CodeV dataset, as the 165k training samples likely overlap or are similar to those in the test set. This raises concerns about overfitting and whether the reported performance truly reflects generalization rather than memorization.\n- There is an inconsistency in GPT-4o’s Pass@1 results on the RTLLM benchmark: this paper reports 33.8, while CodeV-R1 Table 2 reports 56.8. This discrepancy makes it unclear which result is correct and complicates fair comparison. Please confirm which version of RTLLM was used.\n\n- The use of reinforcement learning to improve Verilog generation has already been explored in VeriSeek[1], and leveraging Abstract Syntax Tree-based methods was previously discussed in the “Data is All You Need” [2]. Thus, the novelty of these aspects in this work appears limited.\n\n[1] Wang, Ning, et al. \"Large Language Model for Verilog Generation with Golden Code Feedback.\" arXiv preprint. 2024.\n\n[2] Chang, Kaiyan, et al. \"Data is all you need: Finetuning llms for chip design via an automated design-data augmentation framework.\" DAC. 2024.", "was_revised": true, "final_justification": "I maintain my raw score.", "rebuttal": "**Comments on Weaknesses:**\n\n**W1: A major weakness is the risk of data contamination in the CodeV dataset, as the 165k training samples likely overlap or are similar to those in the test set. This raises concerns about overfitting and whether the reported performance truly reflects generalization rather than memorization.**\n\nWe appreciate the reviewer’s concern regarding potential data contamination. According to the original CodeV [1] paper, a decontamination procedure was applied to mitigate such risks. Specifically, CodeV uses ROUGE-L, a widely adopted similarity metric, to detect overlap between the training and test sets, and removes any training sample with a ROUGE-L score greater than 0.5 relative to the test set.\n\nIn our experiments, we used the same decontaminated dataset and the same base model to train both SAL-V and the CodeV (Qwen2.5) baseline. Despite these identical settings, our method significantly outperforms CodeV (Qwen2.5), which suggests that the performance gain is not due to overfitting or memorization. Rather, it indicates that SAL-V is more effective at learning from the available data and exhibits stronger generalization capabilities.\n\n[1] Zhao Y, Huang D, Li C, et al. CodeV: Empowering LLMs with HDL Generation through Multi-Level Summarization[J]. arXiv preprint arXiv:2407.10424, 2024.\n\n**W2: There is an inconsistency in GPT-4o’s Pass@1 results on the RTLLM benchmark: this paper reports 33.8, while CodeV-R1 Table 2 reports 56.8. This discrepancy makes it unclear which result is correct and complicates fair comparison. Please confirm which version of RTLLM was used.**\n\nThank you for your valuable feedback. We would like to clarify the source of the discrepancy regarding GPT-4o’s performance on the RTLLM benchmark. The Pass@1 score of 33.8 reported in our paper is based on RTLLM v1.1 sourced from CraftRTL [1]. In contrast, CodeV-R1 reports a score of 56.8 on RTLLM v2.0, which is a newer and different version of the benchmark. For reference, CodeV-R1 also reports a Pass@1 of 41.7 for GPT-4o on RTLLM v1.1, which is more consistent with the 33.8 reported in our paper. It’s also important to note that the CodeV-R1 results were released after the NeurIPS submission deadline, and were therefore not available during the preparation of our paper.\n\nFor completeness, we have included our model's performance on RTLLM v2.0 in the appendix. Our method achieves Pass@1 = 56.6, Pass@5 = 68.5, and Pass@10 = 71.7, demonstrating comparable performance to GPT-4o on this more recent benchmark.\n\n[1] Liu M, Tsai Y D, Zhou W, et al. CraftRTL: High-quality synthetic data generation for Verilog code models with correct-by-construction non-textual representations and targeted code repair. arXiv preprint arXiv:2409.12993, 2024.\n\n**W3: The use of reinforcement learning to improve Verilog generation has already been explored in VeriSeek[1], and leveraging Abstract Syntax Tree-based methods was previously discussed in the “Data is All You Need” [2]. Thus, the novelty of these aspects in this work appears limited.**\n\nThank you for the thoughtful feedback. While RL and ASTs have been used in prior works, our approach differs significantly in both motivation and implementation.\n\nVeriSeek [1] uses structural similarity between the ASTs of generated and reference code as the reward. However, such similarity does not guarantee functional correctness, limiting its practical utility. In contrast, our reward is explicitly grounded in functional correctness: we simulate the generated code, identify which output signals behave correctly, and then use AST-based analysis to trace and extract the exact code segments responsible for those signals. This enables fine-grained, signal-level rewards that directly target correct functionality, which is fundamentally more informative than structural matching.\n\nSimilarly, although [2] also uses ASTs, their goal is to translate structural elements of Verilog code into natural language descriptions. Our use of ASTs is entirely different: we perform signal dependency analysis to isolate functionally meaningful code segments for reward calculation. This purpose-driven use of ASTs is novel in the context of reward design for Verilog generation.\n\n[1] Wang, Ning, et al. \"Large Language Model for Verilog Generation with Golden Code Feedback.\" arXiv preprint. 2024.\n\n[2] Chang, Kaiyan, et al. \"Data is all you need: Finetuning llms for chip design via an automated design-data augmentation framework.\" DAC. 2024.\n\n**Reply to Questions:**\n\n**Q1: For Table 1, why does the VeriSeek method achieve such a notably low score on the Specification task?**\n\nThank you for the valuable feedback. The poor performance of VeriSeek on the Specification task is primarily due to its inability to generate correct module headers when they are not provided in the prompt. Unlike other tasks, the Specification task omits the module header, requiring the model to synthesize it purely from the description. VeriSeek often fails in this aspect, likely because it relies on AST similarity as the RL reward signal, rather than functional or syntactic correctness. Without explicitly optimizing for executable or compilable outputs, the model struggles to produce structurally valid modules, especially when critical elements like headers must be generated from scratch.\n\nMoreover, VeriSeek tends to use an outdated Verilog style where input/output declarations and bitwidths are placed inside the module body. In contrast, VerilogEval 1.0 and 2.0 adopt the modern convention of declaring them directly in the module header. This style mismatch often leads to compilation or execution failures during evaluation.\n\nTo verify this, we convert the VeriSeek’s generated module headers to modern style. After the fix, its performance improved (e.g., pass@1: 28.8 for T=0 and 15.9 for T=0.8), but still lagged significantly behind other models. This confirms that the lack of module header generation capability is a key limitation of VeriSeek in this setting."}]

Author rebuttal:

Meta-review: This paper presents SAL-V, a novel framework for Verilog code generation that shifts reinforcement learning optimization from module-level to signal-level rewards. By leveraging AST analysis and signal-aware verification, SAL-V extracts functionally correct code segments from partially incorrect modules, enabling more effective RL training via Direct Preference Optimization. Experiments show state-of-the-art performance on VerilogEval2.0 and RTLLM, with a 7B-parameter model matching much larger models and outperforming open-source baselines.

Reviewers highlighted the method’s originality, strong empirical results, and seamless integration with RL pipelines. Concerns included signal independence assumptions, runtime overhead, and generalizability. The authors’ rebuttal addressed these with empirical validation of signal independence, negligible runtime costs, and updated benchmark results.

SAL-V represents a significant step forward in automated HDL generation. Despite minor limitations, its novelty, technical rigor, and compelling results justify **acceptance**.
Decision: Accept (poster)