VeriLoC: Line-of-Code Level Prediction of Hardware Design Quality from Verilog Code
Authors: Raghu Vamshi Hemadri
Jitendra Bhandari
Andre Nakkab
Johann Knechtel
Badri P Gopalan
Ramesh Narayanaswamy
Ramesh Karri
Siddharth Garg
Venue: NeurIPS 2025 poster ()
Keywords: Hardware Design
LLM Embedding
Congestion
Timing
Primary area: applications

Abstract: Modern chip design is complex, and there is a crucial need for early-stage prediction of key  design-quality metrics like timing and routing congestion directly from Verilog code (a commonly used programming language for hardware design). It is especially important yet complex to predict individual lines of code that cause timing violations or downstream routing congestion. Prior works have tried approaches like converting Verilog into an intermediate graph representation and using LLM embeddings alongside other features to predict module-level quality, but did not consider line-level quality prediction. We propose VeriLoC, the first method that predicts design quality directly from Verilog at both the line- and module-level.  To this end, VeriLoC leverages recent Verilog code-generation LLMs to extract local line-level and module-level embeddings, and trains downstream classifiers/regressors on concatenations of these embeddings. VeriLoC achieves high F1-scores of 0.86-0.95 for line-level congestion and timing prediction, and reduces the mean average percentage error from 14%-18% for SOTA methods down to only 4%. We believe that VeriLoC embeddings and insights from our work  will also be of value for other predictive and optimization tasks for complex hardware design.

VeriLoC: Line-of-Code Level Prediction of Hardware Design Quality from Verilog Code
Raghu Vamshi Hemadri Jitendra Bhandari' Ramesh Narayanaswamy
Andre Nakkab' Ramesh Karri*
Johann Knechtel Siddharth Gargb
Badri P Gopalan
New York University Tandon School of Engineering New York University Abu Dhabi Synopsys
Abstract
Modern chip design is complex, and there is crucial need for early-stage prediction of key design-quality metrics like timing and routing congestion directly from Verilog code commonly used programming language for hardware design). It is especially important yet complex to predict individual lines of code that cause timing violations or downstream routing congestion. Prior works have tried approaches like converting Verilog into an intermediate graph representation and using LLM embeddings alongside other features predict module-level quality, but did not consider line-level quality prediction. We propose VeriLod the first method that predicts design quality directly from Verilog both the line- and module-level. To this end, VeriLoC leverages recent Verilog code- generation LLMM extract local line-level and module-level embeddings, and trains downstream classfiiessregrresors on concatenations of these embeddings. VeriLoC achieves high Fl-scores 0. 0.86-0.95 for line-level congestion and timing prediction, and reduces the mean average percentage error from 14% - 18% for SOTA methods down to only 4%. We believe that VeriLoC embeddings and insights from our work will also be of value for other predictive and optimization tasks for complex hardware design.
Introduction
Modern chip design is highly complex. It begins with devising description of the chip's behavior in hardware description language (HDL) like Verilog.2 This followed by series of automated steps, including synthesis (where RTL code is converted into circuit of Boolean logic and its gate implementation), placement (which arranges gates on the chip canvas), and routing (which connects gates using metal wires). This process transforms the RTL code into manufacturable chip layout.
Key metrics for design quality, like area, timing, power, routing congestion, etc., can only be verified from final layouts, but obtaining these layouts can take hours or days as synthesis, placement, routing, and other steps in the design flow are extremely complex and time-consuming. Designers often iterate multiple times till specifications and quality targets are met; these iterations can take anywhere from weeks to months, impacting time-to-marke Timing and routing congestion, particular, are difficult to manage and are frequently the main impediment to design closure [1-5].
Both authors contributed equally this research. 2DDI. codes are commonly also referred to as register-transfer level (RTL.) descriptions. We use RTL Verilog interchangeably from here
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Time Consuming
RTL Module
Synthesized (AIG)
Layout
257 assign irolword - [eword[23.0].
AND INV
ewordf31-24jj
assign next, Ikey(95554]
1w(1) im(0]
assign next [kay63:332]
[w[2] M 1n[1]:
240 assign next Heey31
iw[3] im(2]
Line with congestion issue
Line level congestion classifier
237 essign rolword (sworr
sword 31:24111
238 assign next. key/96:64)
Line with timing issue
inf1) kw|0];
LLM based verilog encoder
key/63.527
Line level timing classifier
assign next key/31: Q] (n(3) IN[2];
Annotated Module
Line level timing regressor
0.2 ns
Predicted Timing Slack
VeriLoC
Figure I: Conventional flow vs proposed for an exemplary AES design. The RTI. code is converted into a synthesized netlist, e.g., represented by an AIG. @ The netlist is converted into layout, with congestion arising in green areas (bounded in red). @ Congestion information is annotated and traced back to the RTL. With VeriLoC, we directly predict congestion and timing for the RTL at run-time, bypassing the consuming conventional steps.
To address this issue, a body of recent work has proposed early-stage prediction of design quality from RTL code, typically via intermediate representations of RTL like and-inverter graphs (AIGs) [6-10]. However, intermediate representations might lose rich semantic information available in the RTL code compact form. For example, a 64-bit multiplier a single line of Verilog, but corresponds to hundreds or thousands of gates in an AIG, where information must then be reverse engineered by the ML model. Recent work has leveraged large language model (LLM) based encodings of Verilog modules for accurate module-level power, performance, and area (PPA) prediction [11, 12].
Aside from early-stage module-level predictions, designers can greatly from identifying individual lines of code (LoC) responsible for inducing timing violations or routing congestion. While electronic design automation (EDA) tools like RTL Architect [13] provide the capability of aanotaanng the lines of code from final layout (Step 3 in Fig. 1), these too are complex and time-comenn time-consuming Here, we pose a new research question that has not been addressed in literature: can we predict design quality, specifically, timing and routing congestion, from RTL. code at the module and the individual line-of-code level?
A key challenge in addressing this question is how to obtain informative RTL embeddings- -hee, we leverage the recent emergence of LLMs trained specifically for Verilog code generation like CL-Verilog 13B [14]. Although CL-Verilog is decoder-only model, recent studies [15, 16] have demonstrated that internal activations from these models can yield effective embeddings. Using penultimate layer outputs from CL- "L-Veri as embeddings. we propose VeriLoC, novel architecture for live-level classification of Verilog code. To the best of our knowledge, the LoC-level prediction problem has not been addressed in literature before.
The key idea in the proposed VeriLoC architecture (Figure 2) is concatenate embeddings of each line-of-code in a Verilog module with an embedding of the entire module, thus obtaining both a local and global context. In practice, we find that additionally concatenating embeddings from up to two neighboring lines further improves performance. VeriLoC then trains supervised classifier (or regressor) on ground-truth data from Synopsys RTL Architect [17] on the OpenABCD dataset [18],
3 An AIG is a Boolean circuit, which consists only of the so-called universal set of AND and NOT gates, and at the same time functionally identical to the RTL.

---

using models like XGBoost [19] and LightGBM [20] tailored for scarce and imbalanced data. Our contributions are as follows:
We propose and evaluate VeriLoC, a novel LLM--ased architecture for early-stage predic- tion of hardware design quality directly from RTL code, both at the individual lines of code and entire modules. Prior work only performs module-level predictions and converts RTL to intermediate representation, thereby losing rich semantic information.
We identify the importance of capturing both the local context, i.e., neighboring lines of code, and global context, i.e., the entire Verilog module, in enabling line-level timing and congestion predictions. VeriLoC's architecture concatenates both local and global embeddings before the final classification step.
VeriLoC achieves Fl FI-scores of 0.86 in line-level congestion prediction, 0.95 in line-level timing prediction, and also outperforms state-of-art in module-level timing prediction, reducing the mean average percentage error from 18% and 14% to only 4%.
We demonstrate the usefulness of LLMS specialized for RTL code generation to also generate powerful RTL code embeddings that can be used for challenging downstream prediction tasks, specifically, timing and routing congestion prediction.
Overall, VeriLoc' establishes an entirely new approach for early-stage prediction from RTL code, which might be of value not only other prediction tasks, but also for code and design optimization.
2 Background and Related Work
We discuss relevant background hardware design and contrast VeriLoC with related work on predicting design quality and on LLM-based prediction of code quality.
2.1 Hardware Design: Quality Metrics and Prediction
2.1.1 Routing Congestion
What is Routing Congestion? Routing one of the most complex and time-consuming steps in hardware design. Routing entails interconnecting logic gates with wires after the gates are placed on the chip canvas. Modern chips have tens of different routing layers, where any two wires that need to cross without connecting electrically can be routed above another, akin to flyover in traffic network. Almost every problem related to routing is known to intractable [21]. Thus, like most processes in EDA, routing is heavily reliant on heuristic optimization, which cannot guarantee best quality in one go. In this context, managing routing congestion- or congestion for short-is important. This arises when multiple wires pass through the same small area of a chip, such that, in the worst case, the number of routing layers is insufficient to route all wires correctly, i.c., without at least two wires crossing paths. When this happens, the entire design might need to be undergo placement again, or might even necessitate an RTL rewrite.
Predicting Routing Congestion. Traditional methods commonly integrate actual routing pro- cesses [22-25] or analytical models that estimate congestion [26-29]. However, routing-based methods are plagued by considerable runtime cost while analytical-basee approaches suffer from rel- atively low accuracy. To address these challenges, more recent works have employed ML techniques. For example, [30] utilize convolutional neural networks (CNNs) to predict the overall routability of placement solutions. In a follow-up work, [31] employ deep neural networks (DNNs) to achieve better performance for congestion prediction and guide toward congessee placement. Furthermore, [32-37] all use graph neural networks (GNNs) using synthesized and/or placed netlists as inputs, which require running consuming synthesis and placement tools, respectively.
2.1.2 Timing
What is Timing? Timing is critical aspect of chip design and determines the fastest frequency which the chip can operate. Timing is affected by every process in the design cycle, but the most critical impact is within the RTL stage, this dictates the architecture and data flow of the IC. Roughly, timing refers to the time it takes for data to propagate from a circuit's input to its output;
3

---

thus, the longest sequence of gates from the imput to the output is referred to as the critical path. Often, timing is measured by worst negative slack (WNS), i.e., the difference/slack between the desired critical-path delay and the actual critical-path delay. The goal for designers is to push WNS above zero, i.e., to keep delays within the desired budget.
Timing Prediction. Prior works predict timing by employing various ML techniques at various design stages. Closest to our work, [1-4] predict timing for entire modules at the RTL stage, but use either AIGs or other intermediate representations. Of these, [4, 1] are the current statc-of-the- art methods. VeriLoC demonstrates substantial accuracy improvements compared to both. Other methods propose timing prediction at later stages in the design, including after synthesis [5] and after placement [38, 39]. All of these methods are focused on module-level timing prediction; VeriLoC is the first method to provide line-of-code level predictions of WNS.
2.2 LLMs for Code Generation and Quality Prediction
2,2.1 LLMs for Software Code Quality
LLMs have demonstrated significant potential for coding, with applications spanning bug detection, program synthesis, and performance optimization [40]. Models like CodeBERT, GraphCodeBERT, and CodeT5 effectively capture syntactic and semantic nuances in high-level programming languages, making them invaluable for tasks like code summarization, translation, and repair [41, 42). The aforemeenioned LLMs excel at these generative tasks. Bug detection can be viewed as a line-level prediction task, and has been addressed via pattern matching using static analysis [43], enhanced with LLMs [44], or by using LLMs for test generation and fuzzing [45]. In most instances, given the massive amounts of open-source software and vulnerability datasets, these methods can leverage LLMs with careful prompt tuning, retrieval, and agentic frameworks. Indeed, state-of-art approaches prompting to extract structured data-flow paths for bug detection, but do not make any architectural modifications. Unfortunately, data is scarce in hardware, and concepts like routing congestion are barely mentioned. As we show later, prompting methods fail completely for line-level congestion and timing estimation.
like LLMSAN [46] utilize few-shot
2.2.2 LLMs for Hardware
While LLMs originally targeted software code, recent work has extended their use to hardware description languages such as Verilog. Generative Verilog models (e.g., VeriGen [47], CLVerilog [14], RTLCoder [48] and Others [49-52]) achieve impressive synthesis quality but do not provide down- stream quality-of-
(QoR) metrics. Building on this trend, RTLRewriter applies LLMdgd rewriting for optimization [53], and RTLFixer employs LLM--diven debugging to correct syntax errors at scale [54]. Beyond generative tasks, L.MMs have begun to assist QoR estimation for rapid design-space exploration. For PPA estimation, multimodal VeriDistill hardware code with structural or graph-based embeddings to predict power, performance, and area [12, 11]. However, these methods operate at module or graph granularity, leaving line-level semantics unexplored. To the best of our knowledge, VeriLoC is the first ever line-level QoR predic- tor using hardware- speraar---eiaiizeedd LLM, enabling prediction of timing and congestion metrics directly at the statement level in Verilog code.
incluudin CircuitFusion and
3 Methodology
Overview. VeriLoC builds on the premise that LLMs customized for RTL/Verilog code generation that have recently begun to emerge can be also be used as embeddings that capture the semantics of RTL code and used for downstream prediction tasks. VeriLoC is the first demonstrate this property in the hardware context. We illustrate our methodology in Fig. 2. Our approach relies on embeddings generated by CL- CL-Verilog [14], a variant of LLaMA-2 fine-tuned on Verilog code, extracted from its penultimate layer activations. We use these embeddings hierarchically, offering semantic representations at both the module- (Sec. 3.1) and line-level (Sec. 3.2), potentially with more context from neighboring lines (Sec. 3.2). These embeddings are projected to a lower dimension (Sec. 3.3), concatenated and final classifeation/regrsssioo head outputs line- and module-level predictions (Sec. 3.5).

---

RTL Module
Concatenated Line &
Module Embeddings
I.Copprator
Line p-2
Input wire
inpal wen
15:0] Inc.
15:0] int,
Line p-1
CL Verilog
Concatenate
surput reg 0:0] out,
Line
pe]
1 640
ahways (*) begin
out (InO x in1);
Line p+2
1s 10240
5 128
endmodule
Module Embedding
5120
Classifier
Probability Distribution
Softmax
5120 x
CL Verllog Embedding
Logits
Average Pooling
modale L.Companattr
input cik
Decoder Layer
input
Last Hidden State
outout 0:0) out.
input wire 0:0] reset
Embedding Layer
out (in0 in1];
Tokenizer
Attention Mask
und
CL Verilog Input
Detected Congestion
and Timing
Figure 2: The architecture of VeriLoC for line-level timing and congestion prediction from RTL.
Module-level prediction uses module embedding. The context window is to p 5 in this example.
3.1 Module-Level Embeddings
... .......... In), where 1 is the i-th line of code in the module. To capture the global semantics of the module, the complete module is passed through CL. Vervio,,, which generates hidden states H {h1 ht...... ... h,), where h4 & is the latent vector for the i-th token of CL- CL-Verilog's tokenizer, and & the dimensionality of the model's hidden state. Module embeddings e(M) are computed as the pooled dot products of the hidden states and attention mask, normalized by sum of the attention mask:
Modules in RTL designs are represented as sequences of lines of Verilog: M (1 12,12....0
-i=1 (14 m²)
e(M)
>x m,
where m, E {0, 1} is the attention mask that ensures only valid tokens contribute to the embedding, and the dot product h;; my highlights the importance of each hidden state relative to the mask. The resulting module embedding e(M) E & provides a condensed global representation of the module, enabling the detection of macro-level patterns such as resource utilization and timing violations.
3.2 Line-Level Embeddings
To capture localized semantics of Verilog and their impact on design quality, embeddings are also generated for individual lines of code. Each line 4; is passed independently through CL-Verilog, producing hidden state hy. Similar module embeddings, the line embedding e(4, computed using the attention-weighted pooling mechanism:
En (14 ma)
e(4:)
DE m;
These embeddings e(4)) E & focus on the specific performance characteristics of each line, such as whether it contributes congestion WNS.
5

---

3.3 Dimensionality Reduction
After extracting module-level and line-level embeddings, dimensionality reduction applied ensure computational efficiency and improve downstream task performance. The combined embedding J; [e(4;); e (M) undergoes dimensionality reduction as follows." 5 An encoder-decoder framework is trained to reconstruct the original concatenated embedding x; from its reduced representation. The encoder maps the high-dimensional input 2; to lower-dimensional space Rdd: 2; = Encoder(xx)) while the decoder reconstructs и; from 2j: ((z;) Decoder( (nccorer()) The framework is optimized to minimize the reconstruction loss: = || 2 (xx;)l Once training is complete, only the encoder is retained for dimensionality reduction. In short, the encoder provides compact embeddings 2; and serves as a pre-trained initialization for downstream classification and regression.
3.4 Contextual Feature Augmentation
Contextual feature augmentation enhances the representation of a target line by integrat- ing dependencies from surrounding lines. It captures sequential patterns and inter-line re- lationships, which are essential for analyzing RTL code. The embeddings 4 concatenated with those of its neighboring lines within context window p. For any 4, the augmented embedding is: 2aug(4) reduced embedding and [ denotes vector concatenation. This approach introduces lo- cal dependencies, enabling the classifier to capture the sequential nature of RTL designs.
a target line
[zi-pi...; :;; where 2; the
As shown in Fig. 3 (left), the statement always @(posedge clk) begin is not flagged, but in Fig. (right), with saybe_full full <= N17; introduced, the same line always @ (posedge clk) begin becomes congestion-causing. This shows context-aware analysis can enhance detection by considering dependen- cies between neighhouring lines.
(r(N22) begin always (pooeddge dik) begin
always (pasedge dik) begin
ir(N22) bagin
ram[126] <= bts
mayae_ full <= N12 end
end
Congestion
Congestion
Figure 3: Effect of neighbor embeddings in context-aware congestion and timing detection.
3.5
Classification and Regression Heads
The concatenated embeddings feed into classification head for line-level classification of code that cause congestion and timing issues, and a regression head for WNS prediction. We compare three dlssifcaation/regrssson heads: (1) Feedforward Neural Networks (FNNs), a single-layer and fully connected neural network that replaces CL- Verilog's 's original classification head, but with a single classification (or regression) output; (2) XGBoost, a gradient- boosted tree [19]; (3) LightGBM, lightweight
framework optimized for speed and performance [55]. XGBoost and LightGBM are used because of their demonstrated performance on imbalanced datasets [56]. This essential for our work as only small number of lines of code cause congestion or timing issues.
Although our primary focus is LoC-level prediction, also VeriLoC estimate module-level WNS. Specifically, we we first estimate WNS at the line level and then select the worst (smallest) value across all lines. This line-wise granular prediction strategy allows the model to capture granular WNS estimates, and improves upon statc-of-art modulc-level WNS predictors that use only modulc-level embeddings or features. This formulation, to the best of our knowledge, is unique VeriLoC.
Empirical Evaluation
4.1 Experimental Setting
Dataset. We use the popular OpenABCD [18] RTL//erilog code dataset for our experiments, using various Verilog modules from all projects in the dataset. We employed 80/20 random split of the dataset to obtain training vs. test data. Dataset characteristics are shown in Table 1.
Implementation and training detailed in Appendix B.

---

To generate labels for timing and congestion, we use RTL Architect from Synopsys [13], transforming all designs their physical layout, using the open-source Nangate 45nm standard-cell library. We ran all our designs with an aggressive timing constraint of 0. 25 ns, because Synopsys RTL-A only reports WNS when a timing constraints are actually violated.
Table Characteristics of OpenABCD and extracted Verilog. Designs have between 300-30K LOC.
Hyperparameter Setting. For con- gestion and timing detection, we employed XGBoost [19], Light- GBM [20], and an FNN, each tuned to handle class imbalance and opti- mize predictive performance. XG- Boost was configured with default hy- perparameters: scale _aeee_pp__________eeeee_ pos_weight weight set as the ratio of the majority to minority class to mitigate im- LightGBM
Design
# of Modules
of Lines
Design
of Modules
* of Linex
Aea ariane
301
coysta
114
176279
39
214930
dynanic_node node
9
796
black_parrot
88
62948
ethnac
10
1168
bopt
jpog
bp. top
7363
microvatt
26033
op pp_.uiti_tg
89
32959
avery, wragger
57
16496
bp. quad
252
293281
vanilla5
39
11577
balance, max nax_depth=30, used is uss_uballance-True for automatic class weight adjustment, and default settings of num_leave==100, beaninn_rate=0) 05, and feature_frace Erractio=0..8 The regression head followed a similar training procedure using the 'GGBRegressor' with a squared error loss. The FNN consisted of single sigmoid neuron trained with binary cross-entropy (BCE) loss and optimized using Adam with a learning rate of le
peeriinggrate000 .5 and n_estimators=500.
Metries. For congestion and timing classification tasks, we use the F1-score, precision, and recall to measure the balance between sensitivity and specificity. For the regression task of WNS prediction, we employ R and mean absolute percentage error (MAPE), providing insights for goodness-of oftfit and prediction error relative to the target.
Hardware. CL-Verilog feature extraction was performed on a single NVidia H100, and downstream classifiers (XGBoost and LightGBM) were trained/evaluated on CPU machine with 32GB RAM and 8 CPU cores. The FNN model was trained/evaluated using an NVidia RTX 8000 GPU.
4.2 Line-level Classification Results
We begin by discussing VeriLoC's performance on line-level classification for both congestion and timing prediction. Table 2 tabulates our results for three different classification heads, as well as different context lengths (p- [1, 5)).
Congestion Detection. As shown in Table 2, the highest Fi-score for congestion detection is 0.86, achieved by the LightGBM classifier with context length of 5. This result underscores the importance of contextual information in accurately identifying congestion-causing lines.
Timing Detection. The highest F1 Fl-score of 0.95 obtained using the XGBoost and LightGBM classifiers with context length of 5, similar to the best performing model for congestion de- tection. Interestingly, timing prediction results are less sensitive to local context compared to congestion prediction. We achieve Fi-scores of 0.83 for timing prediction even without local context (p 0).
Table 2: Performance of VeriLoC on line-level congestion and timing detection. Best results highlighted in blue.
Classifier
Context Length
Congestion
Timing
P
0
0.38
0.74
0.50
0.67
0.88
0.76
FNN
3
0.41
0.77
0.54
0.71
0.89
0.80
0.86
0.70
0.77
0.76
0.92
0.83
0
0.38
0.74
0.50
0.76
0.92
0.83
XGB
0.42
0.78
0.55
0.91
0.93
These results reinforce prior observations about the advantage of XGBoost and LightGBM over deep networks in handling imbalanced datasets and irregular feature distributions [56], albeit these results were in the context of tabular data. XGBoost and LightGBM improve F1-scores from 0.77 to 0.86 for congestion prediction and 0.83 to 0.95 for timing prediction.
0.94
0.78
0.85
0.94
0.94
0.94
0.82
0.91
LGBM
3
0.41
0.76
0.53
0.93
0.92
0.92
As shown in Table 2, they consistently outperform FNNs in both congestion and timing detection, particularly with a larger context of The highest F1-scores for congestion detection (0.86) and timing detection (0.95) were achieved by these models, reinforcing their robustness in handling
7

---

witne133
bp. muti
cred Douree
-0.10
-0.01
-0.05
-0.30
-0.15
-0.05
-0.30
-0.20
.000
-0.15
-0.2
-0.20
-0.25
-3.00
-0.40
-040
-0.5
-230
-125
-0.3
-035
-3.10
-0.5
-240
-033
-0.3
-925
-0.3
-015
-032
-045
-0.25
-0.30
-2.35
-2.0
-0.35
Adao Viles
Figure 4: Scatter plots of actual vs. predicted line-level WNS using VeriLoC for three Verilog projects. In most instances, the predictions follow the actual WNS closely.
imbalanced datasets. The computational efficiency of XGBoost and its robustmess in optimizing for minority class representation contributed to its superior performance compared to FNNs.
Timing Prediction and Comaprisons with SoTA
Table 3: Line-level prediction of WNS using VeriLoC. Mod-
Table reports VeriLoC's perfor- mance for predicting WNS at the line- level. Across all designs, VeriLoC's achieves high R and low MAPE across all designs but two. The aceu- racy of our WNS predictions are also evident Figure where we compare actual vs. predicted WNS for all lines three benchmarks.
ule level prediction obtained from live-level ones.
Design
R²
MАРE
Design
R²
MAPE
aea
0.97
0.03
coyote
0.99
0.03
ariane
0.96
0.06
dynamic_node
0.76
0.18
black_parrot
0.99
0.01
ethmac
0.96
0.05
top
0.76
0.27
jpeg
0.99
fp_te_o
0.99
0.02
micronatt
0.93
0.21
0.99
0.02
sverv swerv_wrappee _rapper
0.94
0.08
bp_quad
0.98
0.02
vanilla5
0.99
0.02
We now compare VeriLoC's module- level WNS prediction against state-of- art approaches, Recall that VeriLoC's module-level WNS predictions are obtained by first predicting WNS of each line the code and then picking the smallest WNS, while other methods only use entire modulc-level features. Table compares VeriLoC with MasterRTL [4] and RTL-Timer [1], SoTA methods that handcrafted features derived from RTL. Across designs, VeriLoG outperforms prior art, with higher Ra values (always 00.90 and much lower MAPE (always <0.10).
Table 4: VeriLoC vs. SOTA on module-level WNS (timing) prediction, VeriLoG substantially improves Ra and MAPE. Best results are in blue.
Table 5 compares VeriLoC with three additional baselines on module- level WNS: GNN-based predictors from synthesized netlists [10], VeriDistill [11] that uses LLM-based RTL embeddings with GNN-based synthesized look-up-table (LUT) embeddings, and as an ablation, VeriLoC-mod, version VeriLoC (VeriLoC-mod) that only uses module-level but no line-level em- beddings. VeriLoC achieves the best performance, notably improving upon VeriLoC-mod, demonstrating the value of line-level embeddings even for module-level timing prediction.
MT
RTL-Timer
VeriLoC
Design
R²
MAPE
R²
MAPE
R
MAPE
aB3
0.67
0.26
0.76
0.23
0.97
0.08
ariane
0.71
0.15
0.79
0.11
0.93
0.07
black.parrot
0.75
0.10
0.83
0.07
0.99
0.02
bp_te_top
0.76
0.12
0.87
0.08
0.98
0.08
_p_te_top
0.80
0.09
0.88
0.05
0.98
0.04
bp.multi_top
0.61
0.16
0.67
0.13
0.99
0.02
bp_quad
0.69
0.14
0.78
0.10
0.94
0.06
coyote
0.74
0.15
0.83
0.12
0.99
0.03
dynamic_node
0.73
0.20
0.80
0.16
0.99
0.06
ethmac
0.77
0.24
0.83
0.99
0.03
jpeg
0.82
0.29
0.90
0.26
0.98
0.04
microwatt
0.81
0.20
0.80
0.16
0.99
0.02
sverv waraper
0.69
0.27
0.76
0.24
0.95
0.07
VeriDistill is second on MAPE, but performs poorly on R?².
vanilla5 0.68
0.19
0.74
0.16
0.98
0.07
Direct comparisons of VeriLoC with CircuitFusion [12] method, very recent, state-of-art, module-level multimodal PPA predictor, are challenging because they use a different dataset. Still, when comparing the respective improvements upon RTL-Timer, we see that CircuitFusion reports an increase in R² from 0.81 0.83 and reduction MAPE from 16% >> 11%, whereas VeriLoC demonstrates larger relative gains, with R² increasing from 0.86 + 0.94, and MAPE reducing from 12% - 6%. We caution against reading more

---

Table 5: Comparison with SOTA methods for timing prediction on the OpenABCD benchmark. Veri mood uses only module embeddings but no line embeddings.
Metric
GNN [10]
MasterRTL [4]
RTL-Timer [1]
VeriDistill [11]
VeriLoC-Mod
VeriLoC
R
0.69
0.74
0.86
0.728
0.91
0.94
MAPE
0.17
0.15
0.12
0.076
0.08
0.06
into these comparisons because they are on different datasets, and also emphasize that VeriLoC's primary goal of LoC-level predictions is different from CircuitFusion's
4.4
Discussion
We now comment on some interesting properties VeriLoC, avenues for future improvement, and alternate baselines.
Runtime Comparisons. We compare the runtime efficiency of VeriLoC over Synopsys RTL Architect [13] (our synthesis and PnR
CL- CL-Verilog-133 as its base, VeriLoC achieves 14x average speedup and a median speedup of 61 x. To explore the potential for even greater rumtime improvement, we trained 7B CL- Verilog model using the training procedure described in [14]. The 7B model has a x2x on average and 113x median speedup with a modest tradeoff in F1 score of 0.93 for timing and 0.84 for congestion. Thus, while the focus of VeriLoC is accurate line-level PPA prediction, smaller models like the 7B variant or Mixture-of of--exeerts (MoE) based architectures can be adopted to further prioritize inference speed without significant degradation in predictive accuracy.
Impact of Verilog Code Length on Accuracy. To assess the effect of Verilog file length on model performance, we conducted a stratified analysis by splitting test samples based on the number of lines per file. We observed negligible degradation in performance for longer inputs. Specifically, the congestion F1-score dropped slightly from 0.862 for files with fewer than 2000 lines to 0.855 for files with more than 2000 lines. Similarly. the timing F1-score decreased from 0.953 to 0.946, and the Mean Absouute Percentage Error (MAPE) increased from 5% to 6.5%. These results indicate that the model exhibits strong robustness to variations in input length, with only minor performance degradation observed on longer files.
Can Black-Box LLMs Predict Hardware Design Quality? As we have noted before, much of the work in the software community on line-level bug detection uses prompt engineering with pre-trained models. These strategies have been successful because of the abundance of training data in the software world. However, hardware data is scarce, especially so for complex concepts like routing congestion. Thus, we hypothesize that prompting strategies are unlikely to work in the hardware context for livellevel detection of congestion and timing issues. To test this hypothesis, we picked 10 Verilog files from our test dataset and asked ChatGPT-44 [57] to identify lines of code that would cause timing or congestion issues. ChatGPT was unable to identify lines responsible for congestion in any our trials. For timing, ChatGPT identified a line of code correctly in one instance, but also had large number of false positives. Examples of chat responses are given in Appendix D (Fig. 6-9).
5 Conclusion
This paper presents novel M.L--rreee framework, VeriLoC, which provides real-time feedback to hardware designers for the impact of their RTL code on erucial design-quality metrics, timing and congestion. By leveraging embedding from I.IMs, VeriLoC bridges the gap between RTL coding and downstream performance evaluation, enabling designers to make informed decisions early in the design process. Results demonstrate VeriLoC's capability in predicting design metrics at RTL stage, both for individual lines code and at the module level.
Despite its strong performance, VeriLoC has several important limitations. First, it currently operates on leaf modules drawn from the Open-ABCD benchmark and has not been adapted to large-scale industrial designs with inter-module dependencies. Extensive validation on diverse, industry-grade RTL corpora will establish robustness and generalization, but hindered by open access to such data.
Limitations.

---

References
[1] W. Fang, S. Liu, H. Zhang, and 7. Xie, "Annotating slack directly on your verilog: Fine-grained rtl timing evaluation for early optimization," in Proceedings of the 61st ACM/IEEE Design Automation Conference, ser. '24. New York, NY, USA: Association for Computing Machinery, 2024. [Online]. Available:
[2] D. S. Lopera and W. Ecker. "Applying gnns to timing estimation at rtl (invited paper), in 2022 IEFF/ACM International Conference On Computer Aided Design (ICCAD), 2022, pp. 1-8.
[3] D. Sánchez Lopera, Subedi, and W. Ecker, "Using graph neural networks for timing esti- mations of rtl intermediate representations." in 2023 ACM/IEEE 5th Workshop on Machine Learning for CAD (MLCAD), 2023, pp. 1-6.
[4] W. Fang, Y. Lu, S. Liu, Q. Zhang, C. Xu, L. W. Wills, H. Zhang, and Z. Xie, "Masterrtl: A pre-synthesis ppa estimation framework for any rtl design," in Proceedings of 2023 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE, 2023, pp. 1-9.
[5] Z. Xie, R. Liang, X. Xu, J. Hu, C.-C. Chang, J. Pan, and Y. Chen, "Preplacement net length and timing estimation by customized graph neural network," IEEE Transactions on Computer- -ided Design of Integrated Circuits and Systems, vol. 41, 11, pp. 4667-4680, 2022.
[6] A. Hosny, S. Hashemi, M. Shalan, and S. Reda, "Drills: Deep reinforcement learning for logic synthesis," 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC), pp. 581-586, 2019. [Online]. Available: https://api.co https:/app.semanticccllaaaa SwgmmniishhlllooooCopppoo00SSSS.
[7] K. Zhu, M. Liu, H. Chen, Z. Zhao, and D. Z. Pan, "Exploring logic optimizations with reinforeement learning and graph convolutional network," in 2020 ACM/IEEE 2nd Workshop on Machine Learning for CAD (MLCAD), 2020, pp. 145-150.
[8] Y. V. Peruvemba, S. Rai. K. Ahuja, and A. Kumar, "RI-guided runimme-onsstraned heuristic exploration for logic synthesis," in 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD), 2021, pp. 1-9.
[9] A. Basak Chowdhury, B. Tan, R. Carey, T. Jain, R. Karri, and S. Garg, "Bulls-eye: Active few-shot learning guided logic synthesis," IEFE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 42, no. 8, pp. 2580-2590, 2023.
[10] A. B. Chowdhury, M. Romanelli, B. Tan, R. Karri, and S. Garg, "Retrieval-guidee reinforcement learning for boolean circuit minimization," arXiv preprint arXiv:2401. 12205, 2024.
[11) R. Moravej. S. Bodhe, Z. Zhang, D. Chetelat, D. Tsaras, Y. Zhang, H.-L., Zhen, J. Hao, and M. Yuan, "The graph's 's apprentice: Teaching an I1m low level knowledge for circuit quality estimation," 2025. [Online]. Available: https://axxiv.rr//bb///// 1.00843
[12] W. Fang, S. Liu, J. Wang, and Z. Xie, "Circuitfusion:
Multimodal circuit representation learning for agile chip design," in International Conference on Learning Representations (ICLR), 2025. [Online]. Available: hutps//annix.orrgaa//22555.....0
[13] "Synopsys rtl architect," http::/wwwwwoooo.sssscooooooooos http:///attaseeettttrrarrheeeeee
[14] A. Nakkab, S. Q. Zhang. R. Karri, and S. Garg, "Rome was not built in single step: Hierarchical prompting for llmedased chip design," in Proceedings of the 2024 ACM/IEEE International Symposium Machine Learning for CAD, ser. MLCAD '24. New York, NY, USA: Association for Computing Machinery, 2024. [Online]. Available: https//doo.orr/10.. http///11.1111770000 145/3670474.3685966
[15] P. BehnamGhader, V. Adlakha, M. Mosbach, D. Bahdanau, N. Chapados. and S. Reddy, "LIm2vec: Large language models are secretly powerful text encoders," " 2024. [Online]. Available: https://axiv..rr//as//44.....
[16] Y. Luo, T. Zheng. Y. Mu, B. Li, Q. Zhang, Y. Gao, Z. Xu, P. Feng, X. Liu, T. Xiao, and J. Zhu, "Beyond decoder-only: Large language models can good encoders for machine translation," 2025. [Online]. Available: https///arxi..or/////553...66..
10

---

[17] "Synopsys rtl architect," httphh/wwwwww.o.ooooooooooooooop sigmoff/dattshheetstl---ree dttp:///t--arrriiiee-........d
[18] A. B. Chowdhury, B. Tan, R. Karri, and S. Garg, "Openabe-d: A large-scale dataset for machine learning guided integrated circuit synthesis," 2021.
[19] T. Chen and C. Guestrin, "Xgboost: A scalable tree boosting system," in Proceedings of the 22nd aem sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785-794.
[20] G. Ke. Q. Meng. T. Finley, T. Wang. W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, "Lightgbm: a highly efficient gradient boosting decision tree." in Proceedings of the 31st International Conference on Neural Information Processing Systems, ser. NIPS'17. 17. Red Hook, NY, USA: Curran Associates Inc., 2017, p. 3149-3157..
[21] A. B. Kahng, Lienig, I. L. Markov, and Hu, VLSI Physical Design: From Graph Partitioning to Timing Closure. Springer, 2011.
[22] M.-C. Kim, Hu. D.-J. Lee, and I. Markov, "A simplr method for routability-driven placement," in 201 IEEF/ACM International Conference on Computer-Aided Design (ICCAD), 2011, pp. 67-73.
[23] X. He, T. Huang, W.-K. Chow, J. Kuang, K.-C. Lam, W. Cai, and E. F. Young. "Ripple 2.0: High quality routability-drive placement via global router integration," 2013 50th ACM/EDAC/EEE Design Automation Conference (DAC), 2013, pp. 1-6.
[24] W.-H. Liu, C.-K. Koh, and Y.-L.. Li, "Optimization of placement solutions for routability," in 2013 50th ACM/EDAC/EEE Design Automation Conference (DAC), 2013, pp. 1-9.
[25] C.-C. Huang, H.-Y. Lee, B.-Q. Lin, S.-W. Yang, C.-H. Chang, S.-T. Chen, Y.W.. Chang, T.-C. Chen, and I. Bustany, "Ntuplace4dr: A serailedrouutiggdrive placer for mixed-size circuit designs with technology and region constraints," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 37, no. 3, pp. 669-681, 2018.
[26] Spindler and F. M. Johannes, "Fast and accurate routing demand estimation for efficient routability-drrive placement," in 2007 Design, Automation & Test in Europe Conference & Exhibition, 2007, pp. 1-6.
[27] Y. Wei, C. Sze, N. Viswanathan. Z. Li, C. J. Alpert, L. Reddy, A. D. Huber, G. E. Tellez, D. Keller, and S. Sapatnekar, "Glare: Global and local wiring aware routability evaluation," " in DAC Design Automation Conference 2012, 2012, pp. 768-773.
[28] X. He, T. Huang, L. Xiao, H. Tian, and F. Y. Young, "Ripple: robust and effective routability-driverr placer," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 32, no. 10, pp. 1546-1556, 2013.
[29] J.-M. Lin, C.-W. Huang, L.-C. Zane, M.-C. Tsai, C.-L. Lin, and C.-F. Tsai, "Routability- driven global placer target on removing global and local congestion for vlsi designs," " in 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD), 2021, pp. 1-8.
[30] Z. Xie, Y.-H. Huang, G.-Q. Fang, H. Ren, S.-Y. Fang, Y. Chen, and Hu, "Routenet: Routability prediction for mixed-size designs using convolutional neural network," in 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2018, pp. 1-8.
[31] S. Zheng, L. Zou, S. Liu, Y. Lin, B. Yu, and M. Wong, "Mitigating distribution shift for congestion optimization global placement," 2023 60th ACM/IEEE Design Automation Conference (DAC), 2023, pp. 1-6.
[32] S. Yang, Z. Yang, D. Li, Y. Zhang, Z. Zhang, G. Song, and J. Hao, "Versatile multi-stage graph neural network for circuit representation," in Advances in Neural Information Processing Systems, Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc.. 2022, pp. 20313-20324.

---

[33] R. Kirby, S. Godil, R. Roy, and B. Catanzaro. "Congestionnet: Routing congestion prediction using deep graph neural networks," " in 2019 IFIP/EEE 27th International Conference on Very Large Scale Integration (VI.SI-SoC), 2019, pp. 217-222.
[34] B. Wang, G. Shen, D. Li, J. Hao, W. Liu, Y. Huang, H. Wu, Y. Lin, G. Chen, and A. Heng, "Lhnn: lattice hypergraph neural network for vlsi congestion prediction." in Proceedings of the 59th ACM/IEEE Design Automation Conference, ser. DAC '22. New York, NY, USA: Association for Computing Machinery, 2022, p. 1297-1302. [Online]. Available: https://doi.org/100 1ee111...1114404 145/3489517 17.3530675
[35] A. Ghose, V. Zhang, Y. Zhang. D. Li, W. Liu, and M. Coates, "Generalizable cross-graph embedding for gnn-based congestion prediction." " in 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE Press, 2021, p. 1-9. [Online]. Available:
https://doi.org/10.. http://...oorr/oo.111/1///CCCC/ 1199/CCADD110000
2021.9643446
[36] S. Zheng, L. Zou, P. Xu, S. Liu, B. Yu, and M. Wong, "Lay-net: Grafting netlist knowledge on layout-based congestion prediction," in 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD), 2023, pp. 1-9.
[37] K. Back, H. Park, S. Kim, K. Choi, and T. Kim, "Pin accessibility and routing congestion aware DRC hotspot prediction using graph neural network and u-net," in /EEE/ACM International Conference on Computer-Aidee Aided Design (ICCAD), 2022.
[38] Z. Guo, M. Liu, Gu. S. Zhang, D. Z. Pan, and Lin, "A timing engine inspired graph neural network model for pre-routing slack prediction," " in Proceedings of the 59th ACM/IEEE Design Automation Conference, ser. DAC '22. New York. NY, USA: Association for Computing Machinery, 2022, 1207-1212. [Online]. Available: https://doi.on org/ 10. 1111111/34499111111111133 145/34895173353059
[39] P. Cao, G. He, and T. Yang. "Tf-predictor Transformer- prerouting path delay prediction framework," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 42, no. 7, pp. 2227-2237,2022
[40] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, Tillet, F. P. Such, D. Cummings. M. Plappert, F. Chantzis, Barnes, A. Herbert- Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, Achiam, V. Misra, Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, Sutskever, and W. Zaremba, "Evaluating large language models trained on code," CoRR, vol. has/2107.003744 2021. [Online]. Available: https://arxiv. wttp.iv..wrg/aa///1....... 03334
[41] Z. Feng. D. Guo, D. Tang, N. Duan, X. Feng, M. Gong. L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou, "CodeBERT: A pre-trained model for programming and natural languages," in Findings of the Association for Computational Linguistics: EMNLP 2020, T. Cohn, Y. He, and Liu, Eds. Online: Association for Computational Linguisties. Nov. 2020, pp. 1536-1547. [Online]. Available: https://aclanthholos ornnii.nngggg/..nn.iiinnggggg 1mn/p.139/
[42] D. Guo, S. Ren, S. Lu, 7. Feng, D. Tang, S. LIU, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu, M. Tufano, S. K. Deng, C. Clement, D. Drain, N. Sundaresan, J. Yin, D. Jiang, and M. Zhou, "Graphcode (bert): Pre-training code representations with data flow," in International Conference on Learning Representations, 2021. [Online]. Available:
[43] M. Jin, S. Shahriar, M. Tufano, X. Shi, S. Lu, N. Sundaresan, and A. Svyatkovskiy, "Inferfix: End-to-end program repair with llms," in Proceedings of the 31 ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE 2023. New York, NY, USA: Association for Computing Machinery, 2023, p. 1646-1656. [Online]. Available: http:///oo.or///0.114//6/11666.1 1643333331144333388884333
[44] H. Li, Y. Hao, Y. Zhai, and Z. Qian, "Enhancing static analysis for practical bug detection: An llm-integrateed approach," Proceedings of the ACM on Programming Languagss vol. 8, no. OOPSLAI, pp. 474-499, 2024.
12

---

[45] K. Liu, Y. Liu, Z. Chen. J. M. Zhang, Y. Han, Y. Ma, G. Li, and G. Huang, "LIm-powered test case generation for detecting tricky bugs," arXiv preprint XXiv:2404.10304 2024.
[46] C. Wang, W. Zhang, Z. Su, X. Xu, and X. Zhang, "Sanitizing large language models in bug detection with data-flow," in Findings of the Association for Computational Linguistics: EMNLP 2024, Al-Onaizan, M. Bansal, and Y.-N. Chen, Eds. Miami, Florida, USA: Association for Computational Linguistics, Nov. 2024, pp. 3790-3805. [Online]. Available: https///cclantooogy.orgg 0 findings-emnlp.2177
[47] S. Thakur, B. Ahmad, H. Pearce, B. Tan, Dolan-Gavitt, R. Karri, and Garg, "Verigen: large language model for verilog code generation," ACM Trans. Des. Autom. Electron. Syst., vol. 29, no. 3, Apr. 2024. [Online]. Available: http::/doi..or 1rg/100.115/3643611
[48] S. Liu, W. Fang. Y. Lu, Q. Zhang, H. Zhang, and Z. Xie. "Rtlcoder: Outperforming gpt-3.5 in design rtl generation with our open-source dataset and lightweight solution," in 2024 IEEE LLM Aided Design Workshop (LAD). IEEE, 2024, pp. 1-5.
[49] M. Liu, N. Pinckney, B. Khailany, and H. Ren, "Verilogeval: Evaluating large language models for verilog code generation," 2023. [Online]. Available: hutps://axxi.orryabb2330000.....
[50] Y. Lu, S. Liu, Q. Zhang, and Z. Xie, "Rtllm: An open-source benchmark for design rtl generation with large language model," in Proceedings of the 29th Asia and South Pacific Design Automation Conference, ser. ASPDAC '24. IEEE Press, 2024, p. 722-727. [Online]. Available: https://doi.org/ org/100 http:///////////////////////////
[51] Y. Zhao, D. Huang. C. Li, P. Jin, M. Song, Y. Xu, Z. Nan, M. Gao, T. Ma, L. Qi, Y. Pan, Z. Zhang. R. Zhang, X. Zhang, Z. Du, Q. Guo, and X. Hu, "Codev: Empowering Ilms with hdl generation through multi-level summarization."" 2025. [Online]. Available:
htpss//arxiv.or//bb//2/e/ 10424
[52] Z. Pei, H. Zhen, M. Yuan, Y. Huang, and B. Yu, "BetterV: Controlled verilog generation with discriminative guidance," in Proceedings of the 41st International Conference Machine Learning, ser. Proceedings of Machine Learning Research, R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp, Bds., vol. 235. PMLR, 21-27 Jul 2024, pp. 40 145-40 153. [Online]. Available: https:/procceddngs..ll. ptt..eem....eee//2//eeeee......
[53] X. Yao, Y. Wang, X. Li, Y. Lian, R. Chen, L. Chen, M. Yuan, H. Xu, and B. Yu, "Rulrewriter: Methodologies for large models aided rtl code optimization," 2024. [Online]. Available:
http://arxiv.orr/ab//Z/ee 1444 1414
[54] Y.-D. Tsai, M. Liu, and H. Ren, "Rilfixer: Automatically fixing rtl syntax errors with large language models," 2024. [Online]. Available: https//anriv.orgaab//311 11.16543
[55] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, "Lightgbm: A highly efficient gradient boosting decision tree," Advances in neural information processing systems, vol. 30, 2017.
[56] D. C. McElfresh, S. Khandagale, J. Valverde, V. P. C, G. Ramakrishnan, M. Goldblum, and C. White, "When do neural nets outperform boosted trees on tabular data?" in Thirty-seventh Conference Neural Information Processing Systems Datasets and Benchmarks Track, 2023. [Online]. Available: http:://oeaaeiieennn//oooiitia/o
[57] OpenAl, "GPT-4," Mar. 2023. [Online]. Available: https://ppenii.com/rsaaaari
[58] R. I. Bahar, A. K. Jones, S. Katkoori, H. Madden, D. Marculescu, and L. Markov, "Workshops on extreme scale design automation (esda) challenges and opportunities for 2025 and beyond," 2020. [Online]. Available: http:://arri.oorgaa///000.....
[59] X. Wang, Y. Jiang, N. Bach, T. Wang, Z. Huang, F. Huang, and K. Tu, "Automated concatenation of embeddings for structured prediction," in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), C. Zong, F. Xia, W. Li, and R. Navigli, Eds. Online: Association for Computational Linguistics, Aug. 2021, pp- [Online]. Available:
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The central claims and motivation of the paper are articulated in the abstract and reiterated in the concluding paragraph of the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The limitations of this work are addressed in the final paragraph of the conclusion section.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Comprehensive details of the experimental setup, including hyperparameters and dataset specifications, are provided in the results section. Additionally, the code has been made publicly available through open-source release.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (c.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The code required reproduce the results has been open-sourced and is accessible at: wttggghhww..oo/MMMADD/eele......
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm
www.ssssssoolillaoe onPolicyl icy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ wttpppccpppplpapeppppppppppppppp wwtpc//gg//eeeeeeeeeeeeeeesseees
ceabbmmmsssonPPe ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All training and testing details-inclune data splits, hyperparameters, their selection criteria, optimizer type, and related configurations are thoroughly discussed in the 'Experimental Setting" subsection.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
Due to cost considerations we don't do multiple bootstrap runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The necessary information regarding computational resources is provided in the Results section. This includes details on the type of compute infrastructure used, memory specifications, and execution time for the experiments, ensuring that the experiments can be reproduced reliably.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http///eerrrp.eeeee wttpccccpppccceeeeeeeeecccceceet wew.iehhhceeeeeeeeeeeenee
Answer: [Yes]
The research presented in the paper adheres to the NeurlPS Code of Ethics in all respects. It ensures transparency, reproducibility, and responsible use of data and models, with no ethical concerns related to data collection, experimental design, or potential downstream harm.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
17

---

Justification:
The work is for Quality of Results prediction for chip design
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: No such risks arise for this work.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
See Section 4.1. For the Synopsys Tool, have valid license use.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
18

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: See https//gtttaaccom//MAAAAAAeeeeee
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We use a domain-adapted LLM fine-tuned on Verilog to generate embedding,
see Section 3.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: /tops://newue cctpe//e/oeeeeeeec//22223 for what should or should not be described.
20

---

A IC Design
The design of modern ICs with billions of transistors and ever-more complex technology nodes is highly demanding. Thus, sophisticated tooling is essential in this domain known as electronic design automation (EDA). EDA tools are build upon strong notions of abstractions and hierarchical procedures. Next, we outline the industry-standard approach for these procedures. More details relevant for this work are also discussed in subsections in here.
System Specification, Architectural Design. Objectives and requirements for functionality, perfor- mance, and physical implementation are formulated. Modeling languages like SystemC can be used for a formalized approach.
Behavioral and Logic Design. The specification and architecture description are transformed into behavioral model which describes inputs, outputs, timing behavior, etc. for the whole system. Toward that end, specific hardware description languages (HDLs) like Verilog are utilized. The abstraction level for this process is also often referred to as register-transfer level (RTL). To limit design time and efforts, third-party components, so-called IP modules, can be integrated at this stage.
Logic Synthesis. The behavioral model is transformed into a low-level eircuit description, the gate- level netlist (GLN). This step requires technology library for mapping from generic circuit to the technology-specifie circuit that is be manufactured in the end.
Physical Design. The GLN is transformed into an actual physical layout of gates, memories, in- terconnects, etc. Given the high complexity of this stage, it is typically further divided into the following tasks: partitioning and/or floorplanning, power and ground delivery, placement, clock delivery, routing, and timing closure.
Verification and Signoff. The physical layout must be verified against various design and manufactur- ing rules, ensure correct functionality and electrical behaviour. Once all rules are met, the design can be signed-off and taped-out, i.e., send out for fabrication, packaging, and testing.
Practical Challenge. Despite the general success of this compartmentalized approach, a key challenge remains: going through the full EDA stack end-to-end takes considerable time and efforts, in the range of months even for large teams. This challenge is know as the "productivity gap" and is expected stay, especially for ever-more advanced technology nodes [58]. an effort for best design quality, engineers often need to reiterate many times over key processes like placement and routing. Due to the very nature of the hierarchical tooling, the individual processes are often lacking detailed insights from prior stages and, more concerning, reasonable estimates for their impact processes further down in the pipeline. In other words, there is a strong need to integrate well-informed quality assessment into early stages of the EDA pipeline this reiterates the main motivation of this work at hand.
B Encoder-Decoder Architecture for Dimensionality Reduction
This model is designed to effectively compress high-dimensional embeddings into a compact latent space while preserving their semantic content. Below, we analyze structure and key features:
B.1 Network Architecture
The encoder-decoder network consists of fully connected layers with batch normalization, dropout, and non-linear activation functions. The detailed architecture is shown in Table 6.
B.2
Regularization Techniques
To ensure robust learning and prevent overfitting, the following techniques are incorporated:
Dropout: Applied with a 30% probability at multiple layers to prevent co-adaptation of neurons.
Batch Normalization: Normalizes activations at each layer to stabilize training and improve convergence.
LeakyReLUs ReLU: Chosen over standard ReLU to avoid dead neurons and ensure a small gradient for negative values.
21

---

Table 6: Detailed architecture the encoder-decoder network.
Layer Type
Number of Units
Activation Function
Additional Features
Encoder
Fully Connected 4096
LeakyReLU (a 0.01)
BatchNorm, Dropout (30%)
Fully Connected
1024
LeakyReLU (a = 0.01)
BatchNorm, Dropout (30%)
Fully Connected
Latent Dim (d)
LeakyReLU (a = 0.01)
BatchNorm
Decoder
Fully Connected
1024
LeakyReL.U (a = 0.01)
BatchNorm, Dropout (30%)
Fully Connected
4096
LeakyReLU (a = 0.01)
BatchNorm, Dropout (30%)
Fully Connected
Input Dim
B.3 Optimization Strategy
The model is trained using the AdamW optimizer, which combines adaptive learning rates with weight decay regularization. Additional details include:
Learning Rate: Set to a small value (10-4) to ensure gradual convergence.
Reconstruction Loss: The mean squared error (MSE) between the input embeddings and their reconstructed versions is minimized.
Weight Initialization: linear layers are initialized using Xavier Uniform Initial- ization to ensure balanced gradients at the start of training.
B.4 Training Procedure
The model is trained over 200 epochs using mini-batch gradient descent, with the following consider- ations:
Batch Size: A batch size of 128 is chosen to balance memory efficiency and gradient stability.
Orthogonality Regularization: Although not implemented in this iteration, orthogonality constraints on the encoder weights can further enhance disentanglement in the latent space. Validation: Validation loss separate test dataset is monitored to ensure the model generalizes to unseen data.
B.5 Usage Downstream Tasks
The trained encoder produces latent embeddings that serve as input features for classification and regression tasks. These embeddings are compact, noise-robust, and retain essential semantic informa- tion from the original input.
B.6 Justification for Using the Last Hidden Layer VeriLoC
Although VeriLoC focuses on line-level quality prediction, we first conducted complementary ablation study on module-level embeddings to evaluate the predictive quality of hidden states extracted from different layers of the CL-Verrlog model. As shown in Table 7, we observed that embeddings derived from the last decoder layer consistently achieved the highest accuracy across all quality- of-result (QoR) yycliins- the best R² scores and the lowest MAPE for both timing and congestion prediction. These results suggest that the final layer best captures high-level semantic and structural information necessary for downstream design-quality tasks.
Based on this empirical evidence, VeriLoC exclusively uses the last hidden layer extract both line-level and module-level embeddings. This design choice ensures that the model benefits from the richest representation available, without introducing ambiguity or requiring additional architectural tuning select among intermediate layers. Furthermore, the consistently superior performance of the final layer justifies avoiding ensemble or multi-layer fusion strategies, which may add unnecessary complexity without proportional gains.
22

---

Table Prediction quality using embeddings derived from hidden states of the model, specifically
examining the first (Ist, 2nd, 3rd) and last (3rd- Basd,Last, 2nd 2nd-Last, Last) layers for CL- CL-Verilog.
Timing
Congestion
Hidden Layer
R²
MAPE
MAPE
1st
0.66
15.46
0.66
18.13
2nd
0.77
5.64
0.59
23.92
3rd
0.75
6.55
0.72
13.62
3rd-Last
0.85
5.88
0.53
22.34
2nd-Last
0.85
9.32
0.59
17.56
Last
0.89
5.57
0.74
11.66
Table 8: Impact of Hidden Dimension and Classifier Line-Level Detection Performance for Congestion and Timing. Precision (P), Recall (R).
Hidden Dim.
Congestion
Timing
P
R
P
R
32
0.76
0.64
0.77
0.71
64
0.88
0.71
0.86
0.83
XGB
128
0.94
0.78
0.94
0.94
256
0.94
0.78
0.94
0.94
32
0.77
0.64
0.79
0.71
64
0.88
0.72
0.86
0.82
LGBM
128
0.94
0.79
0.96
0.94
256
0.94
0.79
0.96
0.94
C Additional Ablation Studies
C.1 Role of Encoder and Choice of Hidden Dimensions
As noted in Section 3.3, we use an encoder-decoder architecture to reduce the dimensionality of raw line- and module-level embeddings before clasiffcutionn/regression Without the encoder, obtained Fi-scores of less than 0.5. To determine the optimal hidden dimension for the encoder, we study different embedding dimensions across XGBoost and LightGBM. The results are summarized in Table 8.
We find that increasing the hidden dimension significantly improves both congestion and timing detection performance up to a dimension of 128. Beyond 128, however, there is no observable gain in performance, suggesting that further increasing the embedding size is not warranted and that 128 is the optimal hidden dimension, balancing performance and computational efficiency.
C.2 Impact of Comments RTL
As opposed to intermediate representations like AIGs, RTL code also contains comments that hypothesized to be useful in design quality predictions. Here, we evaluate the role of comments in VeriLoC's accuracy. To this end, we conducted study where we removed comments from both the train and test datasets and train VeriLoC classifiers on the resulting code. Table 9 compares line-level detection performance with (w) vs without (w/o) comments show a modest but clear improvement in congestion and timing precision, recall, and F1-scores when comments are included. The benefits are starkest FNN, where FI-scores increase from 0.71 to 0.77 for timing and 0.70 to 0.76 for congestion. XGBoost and LightGBM also benefit from comments, especially for congestion prediction with F1-scores increasing from 0.92 0.95 in the best case.
embeddings. The results
23

---

Table Effect of Comments (C) Module Embedding Generation on Line Detection Performance. Precision (P), Recall (R).
Congestion
Timing
Emb.
P
R
F1
P
F1
w/o C
0.80
0.64
0.71
0.63
0.78
0.70
FNN
w/ C
0.86
0.7
0.77
0.67
0.88
0.76
w/o C
0.93
0.76
0.84
0.91
0.92
0.91
XGB
w/ C
0.94
0.78
0.85
0.94
0.94
0.94
w/o C 0.95
0.77
0.85
0.93
0.91
0.92
LGBM
w/ C
0.94
0.79
0.86
0.96
0.94
0.95
Fig. 5 further supports these findings through saliency-based visualizations of batched vs individual line embeddings. Specifically, the saliency heatmaps reveal that, w/o comments, key structural elements like always ((posedge clk) receive disproportionate amount attention, whereas w/ comments, the model distributes attention more effectively across relevant components.
assign idin
(mmw/cchhddwwt)))cccccccccchhhh din};
// generate multiplier structure
always @(posedge clk)
if(ena)
mult res icoef idin;
Saliency interpretations batched line embeddings
assign idin
[hwwdfhh-wwddhhh]]/c(/((dttt11:1 din);
generate multiplier structure
always @posedgge clk)
(ena)
mult res #1 icoef
idin;
Saliency interpretations of individual line embeddings
Figure 5: Saliency map comparing attention of VeriLoC model (bottom) vs. batched line embeddings (top).
C.3 Alternate Approaches
Recall that including neighboring lines of code can significantly improve performance, especially for congestion prediction. alternate method for including local context, however, would be to obtain a single embedding for batch of consecutive lines B (4-pp..., (+pp by passing

---

B through CL- CL-Verilog as a single input. The model computes batched embeddings Zbatch (B), which natively capture inter-line dependencies via the LLM's attention mechanism.
Although appealing, the batched approach results lower F1-scores, achieving at best an F1 Fl-score of 0.8 for congestion prediction; recall that VeriLoC achieves an F1-score of 0.86. Fig. depicts the attention patterns of the model for congestion detection where the line always @ (posedge clk) is critical. For the batched approach, the model's attention is dispersed across multiple unrelated tokens. Conversely, line-wise embeddings allow the model to prioritize the relevant tokens more effectively, as shown by the darker-red shades. This also aligns with the findings in [59]. which emphasize the importance of modular embedding strategies for structured tasks.
D Case Study Using GPT-4o and VeriLoC
GPT-40
VeriLoC
Prompt: You are an expert in Verilog hardware design. Can you identify the congestion causing lines from the RTL module given below:
module aes rcon(clk, kid, out);
input clk;
module rcok, kid, out);
input kid;
input
clk;
output [31:0]
out;
input
kid;
reg [31:0] out;
output [31:0]
out;
[3:0] rent;
reg [31:0]
out;
wire 13:t_ncntnnexts
reg [3:0]
wire
next:
always ((ooeddge cik)
if(kld) else
out #1 32'h01 _0000_0000___00000 _0_00;
always (posedge dik
out <= #1 frcon(rcnt_nekee) next);
if(kdd else
out #1 32'h01 00;
out <= #1 frron(rcnt_next))
assign next next rent 4'h17 always @ (posedge cik) if(kld) else rent #1 next;
assign rent_nex next rent + 4hh1; always (posedge clk) rent #1 4'h0;
rent <= #1 4'h0;
ir(kld) else
rent <= #1 next,
function [31:0] frcon; input [3:0]
function [31:0] frcon; Input [3:0]
(* parailel case *) case(()
4'h0: frcon=32hh01 00 00_00_00;
(* parallel case *) case())
4'h1: frcon=32h00__0 200020000000000000020 100000000000000000 12.0000000 00;
4'h0: frcon=32'h01 00 00 00:
4'h2: froon=32h04_000000
4'h1: frcon=32hh0__00 00 00 00:
4'h3: frcon=32'h08000 00__0000000000000000000100000000
4'h2: frcon=32'h04 00 00 00)
4'h4: frcon=32'h10 00 00;
4'h3: frcon=32'h08 00.00000000
4'h5: frcon=32'h20 11:0000000000000 00_00;
4'h4: frcon=32/h10 00 00 00:
4'h6: froon=32'h40 000_0000000000000000000000000000
4'h5: frcon=32hh20 00
4'h7: frcon=32'h80 00000000000000000000000000000000
4'h6: frcon=32'h40 00 00 00:
4'h8: frcon=32hhb 100pp_00000000
4'h7: frcon=32h880 003000000000000000000000000 00;
4'h9: frcon=32hh36_00 000__00000000
4'h8: frcon=32'h1b 00 00 00:
default: Tron==22h000D0000__00
4'h9: frcon=32hh36 00 00 00)
endcase endfunction
default: frcon=32h00_000 22220000000000000000 00
000
endcasé
endfunction
endmodule
endmodule
Figure 6: The lines of code highlighted by GPT-4o and VeriLoC for congestion metrics on aes_rcon design.
As part of the case study, understand the capability of generic LLMs to analyze RTL code, took GPT-4o [57] and prompted to report the line numbers in the code responsible for timing and congestion, respectively. We also showcase VeriLoC performance for the same codes. GPT-4o was unable to detect the correct line numbers, i.e, showed false positives, whereas VeriLoC has shown significant results when compared to the ground truth obtained using the EDA tool.
Fig. 6-9 show such examples with highlighted text being the line number reported by GPT-4o and VeriLoC, respectively. Fig. 6 refers to the aes_rcon design, where GPT-4o predicts every line as critical for congestion issues, whereas VeriLoG selectively highlights the correct lines related to congestion. Fig. 7 again shows inability of GPT-4o, now for the task of timing prediction.
25

---

GPT-40
VeriLoC
Prompt: You are an expert Verilog hardware design. Can you identify the timing issue causing lines from the RTL module given below:
module ibx car dk
module iber_car
clk_t.
ret ni,
data
ainon rd dala o.
cata
dasia,,,
ne_error_
rd cenorr
Width
perameter (0:0) ShedkwCopy
perameter
parameter [0:0] ShadowCony
parameter (Width 1100 ReserValue 1'b0::
parameter (Wichh 1:0] ReseWallue 1'sb0;
inputt winz ck_]: inout rst
input reLnni
input wire ckk.
[Width 1:0)
input [Witth-:00] wr_delaji:
oulput wire [Witth- 1:0) dallaajj
input en_t
outpul wire (Widlh- 1:0] rd_dela__o
oulput wire error o;
(Widthh 1:3] roata
output wire emro or
(Width 1:0 rdala_g
ahways @iposedge ok_ negedge rsL
always (ilossedge ni)
negedgers. rsL
rdata_g ResetValus)
ideta_
else (wr_enr])
eise if (wr en_i)
rdata, <= wr data
assign rd od_data_e rdata_gt generato
rdala .q si- St-wr_dlla__
assign generate
(ShadowCopy) begin gen reg [Width 1:0) shacow q: always (iposadge ckk nagadge rst_ni) if (Ite_nn)
(ShadawCopy) bagin gen
reg [Width 1:0] shadow_o
always @lppsedgee ek negedge ret ml)
shadow -ResetValuee
shadow_g (wr _i__]
(wr. [n_
shadio -wr
shadow, q cz -wr data
assign rd. enror rdata q -shadow, q:
assign rd emor = rdata o -shadow
end else begin: shaddwwdwwwww assign enror, 1b0:
end and endgenerate endmodure
gen assign o=
andgenerate
endmadule
Figure 7: The lines of code highlighted by GPT-4o and VeriLoC for timing metries ibex_csr design.
Although it has highlighted the lines related to timing impact, namely the first always block, also reports the next blocks as potential candidates. In Fig. 8, GPT-40 picks registers definitions as potential reason for congestion and skips the actual criticial line containing the additions operation. Finally, in Fig. 9 GP was unable pick the correct always block responsible for timing issues.
26

---

GPT-40
VeriLoC
Prompt: You expert in Verilog hardware design. Can identify the congestion causing lines from the RTL module given below:
[mwidth-1:0] idin; wine [mwidth-1:0] Icoef;
wine [mwidth-1:0] idin; wire [mwidth-1:0] icoef;
reg [mwidth -1:0] rult_res;
wine [nwidth -1:0] ext, mult [e__mutt_rs;
[mwldth -1:0] mult res)
[rwidth -1:0]
//
// module body
//
assign icaef { ((mwidth--width)) (owl:/thhhh) 1])) coef);
module bady
assign idin ((mwicch-dwidth) din din);
(Cmmitth-wwidth)(C (coef cwicth--1 coef)) assign ((wwldtn-dviine d [diiddid 11),, din)i
icoef
// generate multiplier structure
always (pooeddge clk)
I/ generate multiplier structure
if(ena) mult <= icoef Idiin
always @pposedge cik) if(ena) mult
icoef idin;
assign
(3(muit meidtht 1b,, mult res
essign sot_multtres (Bsmmit 1]), mult_res);
// generate adder structure always (pposndge cik) if(ena) if(dcir) result #1 ext muit_reee; else result #1 ext mult res result:
// generate adder structure
always @possddge clk) if(ena)
if(dcrr)
result == é1 ext_mult_res;
eise result #1 ext_mult_res result;
Figure 8: The lines of code highlighted by GPT-4o and VeriLoC for congestion metrics on dct_mac design.
27

---

GPT-40
VeriLoC
Prompt: You an expert in Verilog hardware design. Can you identify the timing issue causing lines from module given below:
module body
always (ppoeedge cik) if(ena)
output [11:0] dout; douten; data-ous
id_zigzag <= #1 dstrb;
assign douten id_zgg integer
variables
always @(posedge cik) Iffena)
reg id_ zigaagg
reg [11:0] sresult [63:0]; // store results for zig zig-zagging
fflld_zigee) // reload results-regisstr begin
// module body
sresuit(63) <= din_00;
always papsedgee di))
in(ena)
sresult(4) <= #1 din 32;
zigrag dstrb)
(44 din 41; sresuit[43] d0n
assign douten zigzag
sresult]33]
always (posedgge dik) (Kena)
sresult din 34;
sresult[31] <= #1 din 43;
IN[ld (igra)) // reload resuits-register file begin
sresult(30 <= #1 din 52;
sresult[27] <= #1 din 71g
3result163] din_00;
sresult(62] <= #1 din_01;
sresuit[19] <- din 36;
sresult[18] <= 81 din 45;
sresult [7] <= #1 din 54;
sresult[16]
din 63
sresuit[
din 72;
sresult[14]] <= din 73:
sresuit[13] <= #1 din din_64;
sresult(09] <= din_47;
sresult[12] <= din_55;
sresuit[08]
din 46;
sresuit[03] din 57;
sresuit [0]
din 37:
sresultf02] <= #1 din din_67:
sresult(09] <= din 47;
sresult(01] <= #1 din_76;
sresult[08] <= #1 din 56;
sresult 003]
din 57:
shift results out
for (n=1: n<=63; n-n+1) // do not change sresult[0]
sresult[01] <= #1 din_76;
sresult[n] <= #1 sresult(n -1];
end else
// shift results out for ns#63,, n=n+ do sresult(0) sresult(n] sresult(n -1];
assign dout sresuit(63];
endinodule
assign dout sresultf631; endmodule
Figure 9; The lines of code highlighted by GPT-4o and VeriLoC for timing metrics zigzag design.
28


Reviews:
[{"review_id": "OO3XwgLmeC", "reviewer": "Reviewer_BE44", "summary": "The authors propose using embeddings from Verilog-code generation LLMs to predict routing congestion and timing violations for hardware designs at the line- and module-level. They demonstrate the performance of their method, dubbed VeriLoC, on the OpenABCD RTL/Verilog code dataset using several metrics, including F1 scores for line-level congestions and timing classification, as well as the $R^2$ and mean-average percentage error for the timing violation regression. They compare with several other SOTA methods, and find VeriLoC outperforms them. They also present additional studies related to the impact of Verilog code length and LLM size.", "questions": "* How reliable are the line- and module-level labels generated for the dataset? How often are congestion and/or timing violations attributed to the wrong line or module? Would another synthesis tool or different settings give (significantly) different labels?\n* I understand the ABCD dataset is public, but will the additional annotations you generated also be made public? This would allow benefit future work.\n* How difficult is it to evaluate CircuitFusion on your annotated ABCD dataset? Is the issue that it would require retraining?", "limitations": "* Authors should address how much they expect the model to generalize, e.g. to different timing constraints or synthesis tools. Would retraining be necessary for small changes? Consider discussing this in the paper or running some additional studies along this line.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths\n* It appears to be the first model addressing the line-level prediction of timing violations and routing congestion, making it quite novel.\n* Comparisons to previous SOTA methods, including MasterRTL and RTL-Timer, for the timing violation prediction, indicating improved performance.\n* Code is public, improving reproducibility and reusability.\n\nWeaknesses\n* No comparison is made to a recent SOTA method called CircuitFusion, where the authors cite difficulty due to the different datasets used for training.\n* It's unclear how well this model can generalize to other synthesis tools or timing constraints.", "was_revised": true, "final_justification": "My score was already a 5: accept. The authors addressed some concerns I had, and I'm happy with their responses. I'm not sure if the paper merits a score of 6 since the scope is somewhat specific to a particular domain, but I hope my original score is high enough that this paper will be accepted.", "rebuttal": "> `Q1`  No comparison is made to a recent SOTA method called CircuitFusion, where the authors cite difficulty due to the different datasets used for training. (+ `Q5`: How difficult is it to evaluate CircuitFusion on your annotated ABCD dataset? Is the issue that it would require retraining?)\n\nThank you for the question! We begin by noting that the main goal/innovation of VeriLoC, line-level prediction, is different from CircuitFusion’s goal of module-level prediction. Therefore, VeriLoC and CircuitFusion cannot be directly compared on this task.\n\nAlthough not our primary goal, VeriLoC and CircuitFusion can be compared on module-level prediction, but these comparisons are challenging because: (1) CircuitFusion's predictive models use not only RTL code, but also code summaries and post-synthesis netlists which would have to be regenerated on ABCD data; (2) three separate transformer models would have to be finetuned on this data. The GitHub implementation for the Circuit was made available only on April 13th 2025, making it difficult to run comparisons in the limited time available before the NeuRIPS submission deadline (the NeuRIPS deadline for citing past work is March 1st).  \n\nFor these reasons, we cite CircuitFusion for completeness but refrain from a direct performance comparison. In fact, **we note that VeriLoC and CircuitFusion make orthogonal contributions: both use embeddings of Verilog modules to which VeriLoC concatenates line-level embeddings, while CircuitFusion concatenates embeddings of code summaries and netlists.** As such, two methods could supercharge each other, but we leave that as a topic of future work and will update the paper accordingly. \n\n---\n\n> `Q2` It's unclear how well this model can generalize to other synthesis tools or timing constraints.\n\nThanks for this excellent question. VeriLoC can be adapted to new synthesis flows and/or timing constraints by retraining the final classification/regression layers on a training set obtained by re-running the backend EDA tool (Synopsys RTL Architect in our case) with the modified flow settings/constraints. Only the prediction/classification layers are retrained, which can be done quickly, while the LLM embeddings are generic and reused.  \n\nTo illustrate this, we **ran a new experiment with a stricter timing constraint of 0.2 ns** as the reviewer requested, resulting in more LoCs with timing and congestion violations. We then retrained the classification/prediction layers on this new data and found that VeriLoC is still able to detect violations with high accuracy. We observe qualitatively similar performance with 0.84 F1-score for congestion, 0.95 F1-score for timing, and 0.94 $R^2$ for WNS Prediction. \n\n---\n\n> `Q3` How reliable are the line- and module-level labels generated for the dataset? How often are congestion and/or timing violations attributed to the wrong line or module? Would another synthesis tool or different settings give (significantly) different labels?\n\nThe ground-truth labels generated are from a commercial EDA tool (Synopsys RTL Architect) that actually performs synthesis and physical design, and then steps backwards from the final result to arrive at line-level annotations. While incorrect annotations cannot be ruled out at such an early stage, we did not observe this in our data. We note that different tool settings do give different results, as noted in our prior response (Q2) regarding stricter timing constraints. \n\n---\n\n> `Q4` I understand the ABCD dataset is public, but will the additional annotations you generated also be made public? This would allow benefit future work.\n\n- Yes, all the source code and dataset generated for this work will be open-sourced."}, {"review_id": "CIunXo6bGK", "reviewer": "Reviewer_ryLD", "summary": "This paper has proposed a workflow using llm to directly extract line-level and module-level information of verilog codes as a predection step to evaluate design quality at the very early stage of chip design. The objective is to predict metrics such as timing, congestion before all the time-consuming backend flows. In this work, the authors proposed using LLM as verilog encoder, followed with various line-level classifier/regressor module for different performance related task. The authors claims the finer line-level information and inter-line level information extractions contribute to higher metric prediciton/classification accuracy.", "questions": "1. As listed in the Weakness.\n2. I also like to ask about the training cost, and the overall cost compared with baseline methods.\n3. I want to know why you chose the exact model architecture?\n4. Do you need to pretrain the encoder or not? Why do you call it LLM when it is just an  transformer model? unless it is pretrained on huge amount of general language data？", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\t1. The encoding strategy prosposed in this paper includes finer granularity. The concatenation of line-level rtl information and module-level information is richer than simple embedding of module rtl\n\t2. The paper has made a good point about hardware design data is very scarce publicly, thus very difficult to reply on pretrained LLM for direct performance prediction for the lack of hugh quality data in pretrain data. And the embedding stage + performance prediction/classification allowing for resource-limited training.\n\t3. The paper has shown sota performance prediction witht he help of finer rtl information over the previous baselines.\n\nWeaknesses:\n\t1. The importance of each single line and how much will it affect the backend metrics such as congestion/timing is still beyond my imagination. For example, in Figure. 4, the author provided visualization of actual vs preducted WNS. But what if the backend EDA algorithms changes, will the strong alignment still hold? Such discussion or proof is missing, which will make the statement less convincing. For example, if the placement & routing algorithm changes or even if the technology nodes changes, will the WNS be the same? And for each line in the RTL, the code itself cleary did not change? How will the model adjust to these without additional information.\n\t2. A following concern is that The model did not provide ablative experiments on the line-level/module-level information. You need to provide it to prove the extra information does help to prediction the features bettern instead of just adopting a stronger model to fit the dataset better.\n\t3. Some more details about the training are not clear to me.  How exactly do you label the data of each line-level? How exacly do you train the encoder? or you just train encoder and decoder at the same time with the label of each module?\n\t4. This is a bit misleading calling it self LLM-based RTL performance prediction. To me, this is just a traditional machine learning based RTL performance prediction with the help from pretrained large language model.", "was_revised": true, "final_justification": "Two new experiments are added to by rerunning the flow with stricter timing constraints and evaluating a model with line-level encodings only without module encodings, which addressed weakness 1 & 2. The other 4 questions are well explained in the rebuttal as well.", "rebuttal": "> `Q1` The importance of each single line and how much will it affect the backend metrics such as congestion/timing is still beyond my imagination. For example, in Figure. 4, the author provided visualization of actual vs preducted WNS. But what if the backend EDA algorithms changes, will the strong alignment still hold? Such discussion or proof is missing, which will make the statement less convincing. For example, if the placement & routing algorithm changes or even if the technology nodes changes, will the WNS be the same? And for each line in the RTL, the code itself cleary did not change? How will the model adjust to these without additional information? ( And the embedding stage + performance prediction/classification allowing for resource-limited training.)\n\nThanks for this insightful question. Indeed, as the reviewer points out, while the prediction/classification layers of VeriLoC are specialised for a fixed EDA flow and technology library (details of the flow and tech library in Sec. 4.1), these layers can easily adapt to new flow parameters or tech. library by retraining these layers alone on a training set obtained by running the backend EDA tool (Synopsys RTL Architect in our case) on Verilog code in the training set with these new settings. Only the prediction/classification layers are retrained, which can be done quickly, while the LLM embeddings are generic and reused.  \n\nTo illustrate this, **we ran a new experiment by rerunning the flow with stricter timing constraints**, resulting in more LoCs with timing and congestion violations. We then retrained the classification/prediction layers on this new data and found that VeriLoC is still able to detect violations with high accuracy. We observe no performance drop with 0.84 F1 for congestion, 0.95 F1 for timing and 0.94 $R^2$ for WNS Prediction. \n\n---\n\n> `Q2` A following concern is that The model did not provide ablative experiments on the line-level/module-level information. You need to provide it to prove the extra information does help to prediction the features bettern instead of just adopting a stronger model to fit the dataset better.\n\nWe appreciate this important observation and draw the reviewer’s attention to Table 5 in the paper that compares “Module-Only” encodings with  “Module + Line-level” encodings. We find that a “VeriLoC-mod” baseline that uses only module-level embeddings performs worse than the full VeriLoC model that uses both line- and module-level features, both in terms of R2 and mean average percentage error (MAPE). In fact VeriLoC-mod is also outperformed by VeriDistill [11], while our full VeriLoC implementation is better than VeriDistill.\n\nWe have also **performed new experiments** evaluating a model with line-level encodings only without module encodings. We observe a  drop in performance in congestion F1-score (drop from 0.86 to 0.54) and timing F1-score (drops from 0.95 to 0.68). We will update the paper with these new results. Appendix C.3 also reports results from an alternate architecture where line-level encodings are computed in “batched” fashion.\n\nThese results confirm that the performance gains are not due to a stronger model alone, but arise from the combination of local (line-level) and global (module-level) semantic information.\n\n---\n\n> `Q3` Some more details about the training are not clear to me. How exactly do you label the data of each line-level? How exacly do you train the encoder? or you just train encoder and decoder at the same time with the label of each module?\n\nThank you for your question. We clarify the two aspects below:\n - **Line-level labeling:** We use Synopsys RTL Architect (RTL-A), a commercial EDA tool that performs full synthesis and place-and-route and then back-annotates timing slack and congestion metrics at the line level, allowing us to assign precise labels for both classification (e.g., congestion/timing issue or not) and regression (e.g., WNS) tasks. \n - **Encoder training:** The encoder is trained as part of an autoencoder architecture, as detailed in Appendix B. Specifically, it learns to compress the concatenated line- and module-level embeddings into alower-dimensional latent space and reconstruct them via the decoder. This training is self-supervised and does not use any labels from RTL-A. After training, only the encoder is retained and frozen. Its latent outputs are then used as features for downstream supervised classifiers and regressors.We hope this clarifies both the labeling process and the encoder training procedure.\n\n---\n\n> `Q4` This is a bit misleading calling it self LLM-based RTL performance prediction. To me, this is just a traditional machine learning based RTL performance prediction with the help from pretrained large language model.\n\nWe appreciate the reviewer’s perspective and agree that our method integrates elements of both LLMs and traditional machine learning. To clarify: VeriLoC leverages a pretrained domain-specific large language model from prior work (CL-Verilog [14], a fine-tuned variant of CodeLLaMA) to extract rich, high-dimensional embeddings of Verilog code. These embeddings, derived from the penultimate layer of CL-Verilog, encode both syntactic and semantic information, and are central to our methodology.\n\nThat said, we understand the concern and **will revise our wording in the final version** to more precisely characterise the method as based on “*LLM-assisted*” since, as the reviewer notes, we are taking help from a pretrained LLM. While recent work ([11,12])  has also used LLMs in the same way, they only do module level predictions while one of our main contributions is an architecture for line–level predictions. We thank the reviewer for highlighting the need for clearer terminology.\n\n---\n\n> `Q5` I also like to ask about the training cost, and the overall cost compared with baseline methods.\n\n- Thank you for the question. We summarize the training and inference costs below, along with a clarification regarding available baselines:\n\n  - Embedding Extraction: We use the pretrained CL-Verilog (13B) model to extract embeddings via a forward pass only (no fine-tuning). This process was performed on an NVidia H100 GPU and took under 2 hours for the full dataset.\n\n  - Encoder Training: The encoder for dimensionality reduction (Appendix B) was trained separately in a self-supervised fashion and converged in ~3 hours on an RTX 8000 GPU.\n\n  - Downstream Training: Classifiers like LightGBM and XGBoost were trained on CPU (8 cores, 32GB RAM), requiring only a few minutes due to the reduced feature dimensionality.\n\n  - Inference Cost: Once embeddings are extracted and the encoder is trained, predictions are extremely fast—enabling 14× to 22× average and up to 113× median runtime speedup over full RTL-to-layout flows using commercial tools like Synopsys RTL Architect, as shown in the \"Runtime Comparisons\" subsection (Section 4.4).\n\n  - Baselines: To the best of our knowledge, no prior work performs line-level prediction of timing or congestion from RTL. Existing methods (e.g., MasterRTL, RTL-Timer, VeriDistill) operate only at the module level, making direct line-level cost or performance comparisons infeasible. This highlights VeriLoC's novelty and the practical importance of its line-level predictions.\n\n- We hope this addresses your concerns regarding training cost and comparative overhead.\n\n---\n\n> `Q6` I want to know why you chose the exact model architecture?\n\nThank you for your question. The choice of the proposed model architecture was guided by extensive experimentation with multiple alternatives and careful analysis of their performance. Key insights that motivated the final design are discussed throughout the paper, particularly in Section 4.3 \"Timing Prediction and Comparisons with SoTA\", Section 4.4 \"Discussion\", and Appendix C \"Additional Ablation Studies\". These sections outline the empirical observations and reasoning that informed our architectural decisions.\n\n---\n\n> `Q7` Do you need to pretrain the encoder or not? Why do you call it LLM when it is just an transformer model? unless it is pretrained on huge amount of general language data？\n\nThank you for raising this important point. For full clarity, as seen in Figure 2, our architecture consists of three components: (i) a pretrained transformer model (CL-Verilog [14]) from which we extract embeddings, (ii) an encoder for dimensionality reduction, and (iii) a downstream classifier or regressor (e.g., LightGBM).\n\n- The LLM component in our method refers specifically to CL-Verilog [14], a Verilog code generation model from prior work which is obtained by finetuning CodeLLaMA on a large corpus of Verilog code obtained from GitHub. We do not further finetune CL-Verilog on our own data, and use it to extract embeddings of our Verilog code from activations of penultimate layers. As noted in our response to `Q4`, we will use the term “LLM-Assisted” for greater clarity.\n \n- We do train that encoder that performs dimensionality reduction as described in our response to `Q3` and detailed in Appendix B and Section 3.3. The final classification/regression heads are also trained on our curated dataset."}, {"review_id": "bZWRHfWbJE", "reviewer": "Reviewer_MVt5", "summary": "This paper proposes VeriLoC, a novel method for predicting timing and routing congestion directly from Verilog code at both the line and module levels. By leveraging LLM-based embeddings, VeriLoC achieves strong performance improvements over prior work. The approach aims to provide early-stage design-quality prediction and may be useful for other hardware optimization tasks.", "questions": "- Why were context windows larger than 5 not tested or reported in Table 2?\n- The timing prediction task is evaluated using binary classification metrics. Could the authors clarify why binary classification was chosen for this task, and what the criteria are for defining positive and negative cases?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "strength:\n- A key strength of this work is its novel focus on predicting design quality at the line level of RTL code, which has not been explored in previous studies. The results also demonstrate that LLM-based embeddings can effectively capture the semantic information needed for accurate line-level quality prediction.\n- The paper is very clearly written, making the methodology and results easy to understand.\n- This work is that the authors have open-sourced their code, which will facilitate reproducibility and encourage further research in this area.\n\nweakness:\n\n- My main concern is the practical applicability of this approach, as directly predicting timing and congestion from RTL code alone is inherently inaccurate. In practice, both timing and congestion are heavily influenced by downstream logic synthesis and physical implementation algorithms, which are not reflected at the RTL level. Therefore, I also find the analysis in Figures 7-9 to be not entirely convincing or clearly justified.", "was_revised": true, "final_justification": "Thank you so much for addresssing my concerns. I will maintain my score. I hope the authors could add more experiment details as they promised.", "rebuttal": "> `Q1` My main concern is the practical applicability of this approach, as directly predicting timing and congestion from RTL code alone is inherently inaccurate. In practice, both timing and congestion are heavily influenced by downstream logic synthesis and physical implementation algorithms, which are not reflected at the RTL level. Therefore, I also find the analysis in Figures 7-9 to be not entirely convincing or clearly justified.\n\n- We completely agree with the reviewer, and note that the impact of synthesis and physical design is implicitly captured by VeriLoC via its ground-truth labels that are obtained from an EDA tool like Synopsys RTL Architect that first performs full synthesis and place-and-route and then back-annotates timing slack and congestion metrics at the line-level. By training against these ground-truth labels, VeriLoC provides quick and early estimation at the line-level without having to go through this time-consuming design flow.\n\n- If the flow parameters or constraints change, these ground-truth labels will change also and the classifier/prediction layers of VeriLoC will be retrained. To illustrate this, we ran a new experiment by re-running the flow with stricter timing constraints, resulting in more LoCs with timing and congestion violations. We then retrained the classification/prediction layers on this new data and found that VeriLoC is still able to detect violations with high accuracy. We observe no performance drop with 0.84 F1 for congestion, 0.95 F1 for timing and 0.94 $R^2$ for WNS Prediction. \n\n---\n\n> `Q2` Why were context windows larger than 5 not tested or reported in Table 2?\n\n- Thank you for the question. We did evaluate larger context windows, specifically p = 7, 9, and 11, during our ablation studies. However, we observed that increasing the window size beyond p = 5 did not lead to any improvement in F1-scores for either congestion or timing prediction. We will update the paper with additional details of these experiments.\n\n---\n\n> `Q3` The timing prediction task is evaluated using binary classification metrics. Could the authors clarify why binary classification was chosen for this task, and what the criteria are for defining positive and negative cases?\n\n- Thank you for the question and apologies if this was not clear. The baseline commercial EDA tool that generates ground-truth labels only reports which lines exceed/violate a given timing or congestion constraint, since it is typical in industry to focus on focus on these parts of the design. Thus we set up a binary classification task where we assign a positive label to a line if its WNS is it violates a specified timing/congestion constraint and a negative label otherwise.  \n- The tool also provides the actual worst negative slack (WNS) per line for lines that violate timing which we formulate as a regression problem and report results in Tables 3-5."}, {"review_id": "UsFwwP98wc", "reviewer": "Reviewer_a3PJ", "summary": "This paper proposes a method for predicting hardware design quality metrics (timing violations and routing congestion) by combining LLM-based Verilog code embeddings with tree-based models (XGBoost/LightGBM).", "questions": "- How does the model perform on designs with different coding styles (e.g., generated RTL vs. handwritten)?\n- Are there plans to integrate this into industry workflows (e.g., as a plugin for Vivado/VSCode)? What are the inference latency and memory overheads?\n- How would NeurIPS audiences be interested in this? I honestly suggest to submit to EDA venues (DAC, DATE, ICCAD, etc.).", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is well-structured, and figures (e.g., pipeline overview) effectively illustrate the workflow.\n- The methodology is technically sound, with thorough ablation studies (e.g., dimensionality reduction, embedding granularity) supporting design choices.\n- Addresses a real-world pain point in RTL design verification, potentially reducing EDA tool runtime.\n\nWeaknesses:\n- Straightforward application of embedding LLM. LLM + dimensionality reduction + classifier is not fundamentally novel outside hardware contexts, adding little value to the field of neural information processing.", "was_revised": true, "final_justification": "I would like to keep my positive assessment. This is a good paper discussing on a valuable and interesting problem. The method is a bit flat in flavor but that is ok. I do acknowledge the alignment of topics after rebuttal.", "rebuttal": "> `Q1` Straightforward application of embedding LLM. LLM + dimensionality reduction + classifier is not fundamentally novel outside hardware contexts, adding little value to the field of neural information processing.\n\nFrom a machine learning perspective, we believe that our *VeriLoC architecture is the first to blend both line-level and module-level embeddings for line-level classification*, and as such *might also be useful for line-of-code level software prediction tasks* (e.g., line-level bug prediction/dectection). We bring in new observations including the role of local context around each line of code and its importance in achieving state-of-art performance. There is also a growing body of work at ML conference on ML/AI for hardware design (please see some references below) since it is an area where ML/AI is having major impact, and we hope to contribute to this body of work.  \n\n---\n\n> `Q2` How does the model perform on designs with different coding styles (e.g., generated RTL vs. handwritten)?\n\nThanks for the excellent question. Our experiments thus far are on human-written RTL code, but could be replicated on generated RTL, for example, from a high-level synthesis tool. While we could not run these experiments in the rebuttal timeframe, we will add this as a topic for further study. \n\n---\n\n> `Q3` Are there plans to integrate this into industry workflows (e.g., as a plugin for Vivado/VSCode)? What are the inference latency and memory overheads?\n\nThanks for the great question! Yes, we are investigating integration into industry workflows. On average, inference takes ~5.6 seconds per Verilog module when using the 13B CL-Verilog model. The end-to-end process, including embedding extraction, encoding, and prediction, requires up to 26 GB of GPU memory. We also trained a 7B CL-Verilog variant, which reduces memory usage and latency to 3.5 seconds with only minor accuracy tradeoffs (see Section 4.4).\n\n---\n\n> `Q4` How would NeurIPS audiences be interested in this? I honestly suggest to submit to EDA venues (DAC, DATE, ICCAD, etc.).\n\nThere is a growing body of work published in leading AI/ML conferences on AI/ML for hardware design and the novel challenges this application domain brings (see references [1]-[5] below as examples).  The CircuitFusion paper on module-level timing prediction was also recently published at ICLR’25.  As such, we do think NuerIPS attendees would be interested in this topic of growing relevance. Furthermore, the proposed VeriLoC architecture might be useful in software prediction tasks like line-level bug detection, and therefore, we might have interest from the \"AI for software coding\" community as well.\n\n\n---\n\n`References:`\n\n[1] Wenji Fang, Shang Liu, Jing Wang, & Zhiyao Xie (2025). CircuitFusion: Multimodal Circuit Representation Learning for Agile Chip Design. In The Thirteenth International Conference on Learning Representations.\n\n[2] Animesh Basak Chowdhury, Marco Romanelli, Benjamin Tan, Ramesh Karri, & Siddharth Garg (2024). Retrieval-Guided Reinforcement Learning for Boolean Circuit Minimization. In The Twelfth International Conference on Learning Representations.\n\n[3] Vasudevan, S., Jiang, W., Bieber, D., Singh, R., shojaei, h., Ho, C., & Sutton, C. (2021). Learning Semantic Representations to Verify Hardware Designs. In Advances in Neural Information Processing Systems (pp. 23491–23504). Curran Associates, Inc..\n\n[4] Zehua PEI, Huiling Zhen, Mingxuan Yuan, Yu Huang, & Bei Yu (2024). BetterV: Controlled Verilog Generation with Discriminative Guidance. In Forty-first International Conference on Machine Learning.\n\n[5] Yang, S., Yang, Z., Li, D., Zhang, Y., Zhang, Z., Song, G., & Hao, J. (2022). Versatile Multi-stage Graph Neural Network for Circuit Representation. In Advances in Neural Information Processing Systems (pp. 20313–20324). Curran Associates, Inc.."}]

Author rebuttal:

Meta-review: This paper introduces VeriLoC, the first framework to predict hardware design quality at both the line-level and module-level directly from Verilog, addressing both congestion and timing. The paper makes a clear and original contribution to early-stage hardware design quality prediction, extending LLM-based techniques to the novel and practically relevant line-level setting. In the rebuttal discussion, the author made clear clarifications, added new experiments with stricter timing constraints, ablation studies, and training/inference cost. The reviewers agreed that their major concerns were resolved and are all in favor of acceptance. I recommend acceptance, and hope the authors could incorporate the additional experimental results and clarifications into the camera-ready version.
Decision: Accept (poster)