Title: QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Verilog Code Generation

Code Generation

LLM
Abstract: The remarkable progress of Large Language Models (LLMs) presents promising opportunities for Verilog code generation which is significantly important for automated circuit design. The lacking of meaningful functional rewards hinders the preference optimization based on Reinforcement Learning (RL) for producing functionally correct Verilog code. In this paper, we propose Signal-Aware Learning for Verilog code generation (QiMeng-SALV) by leveraging code segments of functionally correct output signal to optimize RL training. Considering Verilog code specifies the structural interconnection of hardware gates and wires so that different output signals are independent, the key insight of QiMeng-SALV is to extract verified signal-aware implementations in partially incorrect modules, so as to enhance the extraction of meaningful functional rewards. Roughly, we verify the functional correctness of signals in generated module by comparing with that of reference module in the training data. Then abstract syntax tree (AST) is employed to identify signal-aware code segments which can provide meaningful functional rewards from erroneous modules. Finally, we introduce signal-aware DPO which is optimized on the correct signal-level code segments, thereby preventing noise and interference from incorrect signals.
The proposed QiMeng-SALV underscores the paradigm shift from conventional module-level to fine-grained signal-level optimization in Verilog code generation, addressing the issue of insufficient functional rewards.
Experiments demonstrate that our method achieves state-of-the-art performance on VerilogEval and RTLLM, with a 7B parameter model matching the performance of the DeepSeek v3 671B model and significantly outperforming the leading open-source model CodeV trained on the same dataset.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a signal-aware learning framework (SAL-V) for Verilog code generation, using AST-based extraction of functionally correct code segments to provide fine-grained RL rewards. The approach addresses limitations of module-level rewards and achieves state-of-the-art results on multiple benchmarks, showing strong performance even with smaller models.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n- Employing abstract syntax tree (AST) analysis to identify signal-aware code segments is a novel approach for data extraction, and it is well-suited to Verilog code generation. This represents an interesting problem and a promising direction for further exploration.\n- The experimental results validate the effectiveness of the proposed method across multiple datasets, demonstrating its robustness and broad applicability.\n\nWeakness:\n\n- A major weakness is the risk of data contamination in the CodeV dataset, as the 165k training samples likely overlap or are similar to those in the test set. This raises concerns about overfitting and whether the reported performance truly reflects generalization rather than memorization.\n- There is an inconsistency in GPT-4o’s Pass@1 results on the RTLLM benchmark: this paper reports 33.8, while CodeV-R1 Table 2 reports 56.8. This discrepancy makes it unclear which result is correct and complicates fair comparison. Please confirm which version of RTLLM was used.\n\n- The use of reinforcement learning to improve Verilog generation has already been explored in VeriSeek[1], and leveraging Abstract Syntax Tree-based methods was previously discussed in the “Data is All You Need” [2]. Thus, the novelty of these aspects in this work appears limited.\n\n[1] Wang, Ning, et al. \"Large Language Model for Verilog Generation with Golden Code Feedback.\" arXiv preprint. 2024.\n\n[2] Chang, Kaiyan, et al. \"Data is all you need: Finetuning llms for chip design via an automated design-data augmentation framework.\" DAC. 2024.\n\nQUESTIONS:\nFor Table 1, why does the VeriSeek method achieve such a notably low score on the Specification task?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SAL-V, a novel signal-aware learning approach for Verilog code generation. The key idea is to leverage functionally correct output signal code segments to optimize RL training. The method addresses the challenge of insufficient functional rewards in LLM-based Verilog generation by identifying and utilizing correct signal-level implementations from partially incorrect modules. It introduces a signal-aware DPO that focuses on correct signal-level code segments. The approach achieves state-of-the-art performance, demonstrating a significant advancement in the field.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper introduces a new paradigm for Verilog code generation, shifting from module-level to fine-grained signal-level optimization. This is a crucial innovation that addresses a fundamental limitation of existing RL-based methods, which struggle with sparse functional rewards.\n\n\n2. SAL-V's ability to extract meaningful functional rewards from erroneous modules by verifying and isolating correct signal implementations is highly impactful. The use of AST for signal-aware code extraction is a clever technical contribution.\n\n\n3. The experimental results are compelling, showing that SAL-V achieves competitive and even superior performance compared to much larger models (e.g., matching DeepSeek-V3 671B with a 7B model) on challenging benchmarks like VerilogEval2.0 and RTLLM.\n\n\nWeaknesses:\n1. While the proposed signal-aware learning is highly effective, it would be beneficial to see the methodology validated on large-scale on-policy RL. The potential of off-policy RL for very large-scale training might be limited, and demonstrating the applicability and scalability of SAL-V within an on-policy framework would further strengthen the paper's claims and implications for future research.\n\nQUESTIONS:\nPlease refer to the weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SAL-V, a reinforcement learning (RL) approach to Verilog code generation, emphasizing fine-grained signal-aware rewards and Direct Preference Optimization (DPO). Unlike conventional methods that use module-level rewards, SAL-V employs signal-level correctness rewards extracted through Yosys AST parsing and dependency analysis, guiding the generation of Verilog code. Experimental evaluation demonstrates significant performance improvements on established benchmarks, surpassing several strong baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The proposed signal-level correctness reward approach significantly refines traditional module-level RL feedback, potentially offering better learning signals.\n2. Evaluations across VerilogEval2.0 and RTLLM benchmarks demonstrate the method’s superior performance compared to strong baselines such as GPT-4o and DeepSeek-v3.\n3. DPO provides more stable training compared to PPO, aligning effectively with the structured and fine-grained nature of Verilog code.\n4. Detailed ablation studies demonstrate the contributions of various components of the proposed method.\n\nWeaknesses:\n1. The baseline models' details, such as their specific architectures and training setups, are not clearly articulated. The paper does not explicitly state or adequately discuss the choice of base models.\n2. Presentation Issues: Some typographical issues were observed, including inconsistent decimal alignment in Table 1 for \"CodeV(Qwen1.5)\" and suboptimal sizing of Figure 2(a), making the framework overview challenging to interpret clearly.\n3. Incomplete Related Work Discussion: While several recent works have explored RL-based RTL generation, such as \"Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback,\" the authors do not provide comparative analysis with these related approaches. Given the lack of theoretical justification and limited experimental settings, omitting such comparisons makes the conclusions regarding the superiority and necessity of the signal-aware method less convincing.\n\nQUESTIONS:\n1.\tCould you clarify which base models were used to train SAL-V and how they were configured (e.g., frozen layers, RL steps, DPO parameters)?\n2.\tWhy were simpler RL strategies like module-level reward (e.g., from Insights from Verification) not included for comparison?\n3.\tHow does your signal extraction pipeline handle ambiguous or non-standard Verilog code?\n4.\tIs the Yosys-based extraction scalable to large, complex modules?\n5.\tCould you provide a brief overview of the distribution of your signal-aware dataset—how many signals, what kinds, and how are preference pairs balanced?\n\nLIMITATIONS:\nThe authors partially acknowledge limitations, but further elaboration is necessary on failure modes of signal-level extraction, scalability, and comparisons with simpler methods.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA key challenge in RL-based code generation for hardware is the lack of good rewards, existing approaches rely on structural similarity between generated and reference code, which doesn’t always reflect functional correctness. SAL-V tackles this by shifting the optimization focus from the entire module level to the level of individual output signals. they extract these correct signal segments by checking signal correctness via simulation and identifying the corresponding AST subtrees. Their approach leads to stronger and more targeted learning signals during RL\n\nSTRENGTHS AND WEAKNESSES:\n- Signal-level reward formulation is novel and well-justified,  shifts RL optimization from module-level to signal-level using functionally verifiable outputs, enabling learning from partially correct generations.\n- Seamless integration into Direct Preference Optimization training. The method doesn’t require architecture changes, making it easy to plug into existing SFT+DPO pipelines for code generation.\n- Assumption of signal independence is untested, no experiments probe scenarios with dependent or correlated signals, which are common in real designs, is it a safe assumption? All evaluations are conducted on curated, synthetic datasets.\n- No runtime analysis is presented to assess overhead of simulation and AST parsing.\n- Comparison with alternative reward schemes?\n\nThe paper presents a creative, well-motivated, and technically sound method for improving reward fidelity in Verilog code generation. However, the paper would benefit from more thorough analysis in several areas—robustness to signal dependencies, runtime costs, qualitative behavior, and evaluation beyond curated benchmarks. These limitations suggest that while the core contribution is strong, its generalization and practical deployment may still require further work.\n\nQUESTIONS:\nCan you report training cost and overhead introduced by simulation + AST parsing?\nCould you compare signal-wise rewards to other fine-grained alternatives?\nIs SAL-V effective when used with other RL algorithms beyond DPO?\nCan you include qualitative examples and error analysis of signal-wise segment extraction?\nHow does SAL-V perform when signal outputs are interdependent?\nHow reliable is AST extraction when the generated code is invalid or malformed?\n\nLIMITATIONS:\nyes in appendix.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}