DynamicRTL: RTL Representation Learning for Dynamic Circuit Behavior

Authors: Ruiyang Ma, Yunhao Zhou, Yipeng Wang, Yi Liu, Zhengyuan Shi, Ziyang Zheng, Kexin Chen, Zhiqiang He, Lingwei Yan, Gang Chen, Qiang Xu, Guojie Luo
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Code Representation Learning, Graph Neural Network, Hardware Design

Abstract:
There is a growing body of work on using Graph Neural Networks (GNNs) to learn representations of circuits, focusing primarily on their static characteristics. However, these models fail to capture critical runtime behavior, which is crucial for tasks like hardware verification and optimization. To address this limitation, we introduce DynamicRTL, a novel GNN-based approach that learns circuit representations by incorporating both static structures and multi-cycle execution behaviors. DynamicRTL leverages an operation-level Control Data Flow Graph (CDFG) to represent Register Transfer Level (RTL) circuits, enabling the model to capture dynamic dependencies and runtime execution. To train and evaluate DynamicRTL, we built the first comprehensive dynamic circuit dataset, comprising over 6,300 Verilog modules and 190,000 simulation traces. Our results demonstrate that DynamicRTL consistently outperforms existing models in branch prediction tasks. Furthermore, its learned representations transfer effectively to related tasks, achieving strong performance in assertion prediction and underscoring its transfer learning capabilities for dynamic circuit tasks.

Decision: reject
Reviewer scores: 5, 6, 5, 3, 6

Reviews and discussion:
<thread>
creation date:
1729994939474

thread_type:
official_review

<thread_object>
creator: reviewer_autv

<content>
summary:
The DynamicRTL paper introduces a Graph Neural Network model that captures dynamic behaviors in RTL circuits. Unlike traditional models that focus on static structures, DynamicRTL uses Control Data Flow Graphs (CDFGs) to incorporate both static and dynamic data, enabling more accurate branch execution and assertion predictions. The model achieves good accuracy in branch prediction and demonstrates strong transfer learning potential for tasks like assertion prediction. DynamicRTL offers insights for hardware verification and optimization, setting a new standard for understanding dynamic circuit behaviors.

strengths:
The DynamicRTL paper introduces an approach that combines static and dynamic circuit information. The model delivers results that outperforming other methods in tasks like branch prediction and assertion prediction, while also contributing a dynamic circuit dataset. The paper is clear in its presentation, offering explanation of the CDFG construction and the DR-GNN model’s architecture. Lastly, DynamicRTL tackles a significant problems for the evolving field of hardware design and electronic design automation (EDA).

weaknesses:
The weaknesses of the DynamicRTL paper are:
it relies heavily on existing GNN methods and concepts like CDFGs, making it less groundbreaking in terms of underlying technology. 
The model’s generalizability to other circuit types or levels, such as netlists, remains unexplored, and its performance on larger or more complex circuits could be tested more rigorously. 
Additionally, the paper’s significance is limited, as it primarily targets RTL-level circuits, restricting its applicability to broader hardware contexts or higher abstraction levels, such as layout optimization tasks.

questions:
The paper would benefit from including a brief mention of the results in the abstract to give readers a clearer overview of its performance. Additionally, it would be useful to know how the proposed design compares with existing commercial tools in terms of efficiency and accuracy. Finally, more details on how input sequences are selected for testing would be helpful to understand the robustness of the model in different scenarios.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730094590436

thread_type:
official_review

<thread_object>
creator: reviewer_df1p

<content>
summary:
This work introduces DR-GNN, a GNN-based model for learning circuit representations via incorporating dynamic information. Specifically,  DR-GNN integrates static structures with multi-cycle execution dynamics using a Control Data Flow Graph (CDFG) representation of RTL circuits. Trained on a newly developed dataset of over 6,300 Verilog modules and 190,000 simulation traces, DR-GNN achieves promising results on branch prediction tasks compared to conventional GNN models. Additionally, it demonstrates transferability to related tasks like assertion prediction.

strengths:
- Overall, the paper is well written. All technical steps are easy to follow.
- Learning dynamic circuit representation is both an interesting and important problem.

weaknesses:
- Limited Novelty: As the authors noted, the proposed model architecture is largely based on the Heterogeneous Graph Transformer [1], and the use of GRU has also been explored in numerous prior works (e.g., [2]). Therefore, I regret to say that the novelty of this work is insufficient.
- Missing Stronger and More Relevant Baselines: Several recent circuit representation learning methods (e.g., [3-5]) are omitted from the experiments. Notably, [3] also proposes learning different aggregation functions for different node types. Comparing only with basic GNNs like GCN does not fully convince me of the proposed approach's effectiveness.
- Unconvincing Claims: While the authors claim that DR-GNN overcomes the over-smoothing issue, Figure 4 shows a significant drop in accuracy with 30 layers. Additionally, there are numerous prior works addressing over-smoothing [6], yet they are discussed without comparison in this study.
- Limited Reproducibility: The manuscript lacks detailed hyperparameter settings for DR-GNN and baseline GNNs, making it unclear how hyperparameters were tuned. More importantly, since the source code and generated circuit dataset are not provided, it is nearly impossible to reproduce the results presented in this work.

[1] Hu et al., "Heterogeneous Graph Transformer", WWW'20. \
[2] Li et al., "Gated Graph Sequence Neural Networks", ICLR'16. \
[3] Wang et al., "Functionality Matters in Netlist Representation Learning", DAC'22. \
[4] Yang et al., "Versatile Multi-stage Graph Neural Network for Circuit Representation", NeurIPS'22. \
[5] Deng et al., "Less is More: Hop-Wise Graph Attention for Scalable and Generalizable Learning on Circuits", DAC'24. \
[6] Rusch et al., "A Survey on Oversmoothing in Graph Neural Networks", arXiv'23.

questions:
- It appears the authors split the circuit dataset into only training and test sets, with no validation set. How was hyperparameter tuning performed in this case?
- What is the variance in accuracy scores reported in Tables 1 and 3?
- How does the model’s accuracy vary across circuit designs with different depths (in contrast to different sizes in Table 2)?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730554396024

thread_type:
official_review

<thread_object>
creator: reviewer_wxwe

<content>
summary:
The authors propose a representation learning approach for RTL descriptions of circuits. Unlike prior works that generally learns the static structure of the circuits, this work is capable to cover the dynamic execution features of circuits which change with inputs and circuit states. Specifically, as  the circuit’s functionality is embedded in the attributes of the nodes and their connectivity, the authors model the RTL as an operational-level CDFG and apply GNNs to learn circuit representations. Notably, because some operator nodes in circuits are non-commutative (e.g., subtraction), the authors parameterize the weights of the heterogeneous attention based on different operation types and node positions, enabling effective aggregation of different circuit operations. Finally, the authors verify the proposed GNN model with two downstream tasks.

strengths:
- This work targets at representation of circuits with dynamic behaviors for the first time.

- This work addresses the over-smoothing problem encountered in the GNN model.

- This work also builds a benchmark for dynamic circuit modeling.

weaknesses:
1) The dynamic characters of the circuit mentioned in this work mainly include branch and multi-cycle execution. Do they cover all the dynamic features appropriately? It needs to be defined more clearly and formally. For instance, there are many iterative algorithms such as equation solvers implemented with hardware, but the number of iterations depends on the amount of the error and it affects the runtime and power consumption of the circuit substantially. Can it be estimated given the initial inputs using the proposed GNN model?  

2) The experiments are relatively weak. The basic goal of representation of circuits with dynamic characteristics is PPA estimation, but it is not mentioned nor compared to prior works. The proposed downstream tasks are not quite typical tasks. In addition, the circuits used by the authors (with node counts ranging from 10 to 500) are too small which is insufficient to validate the proposed approach.

questions:
See the weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730614862367

thread_type:
official_review

<thread_object>
creator: reviewer_q4c3

<content>
summary:
The paper focuses on improving upon the current state of GNNs for circuit design that depend on the static data of circuits by including dynamic behaviors. The paper creates a large dataset with various designs and simulation traces, then build a dynamic RTL GNN model that is built on the circuit operation-level graph. The paper shows the effectiveness of this representation on branch prediction, then shows that this can be transferred to assertion prediction task.

The paper seems to improve by exploring an underexplored dimension of dynamism in circuit-GNNs. The paper is well organized and shows that it can perform well on a given task + transferred to a downstream task that hints the effectiveness of this new representation and of adding dynamism to the circuit-GNNs.

strengths:
The paper seems to improve by exploring an underexplored dimension of dynamism in circuit-GNNs. The paper is well organized and shows that it can perform well on a given task + transferred to a downstream task that hints the effectiveness of this new representation and of adding dynamism to the circuit-GNNs.

weaknesses:
The task that was used seems to be too simple to say that it is effective. Seems like the tasks are simply binary classification tasks which is not sufficient.

questions:
* Can you provide more details about the test set for the two tasks? For example, size of the dataset, is it balanced, ... Without that information, it is possible for the data to be biased.

* It seems that the number of nodes in the circuit are too small. How does this scale for much larger ones? At what point does the accuracy drop significantly for table 2.

* How is the speed of inference and training?

* For the two tasks, how does it compare to human evaluation or any pre-existing programs?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730752958393

thread_type:
official_review

<thread_object>
creator: reviewer_km46

<content>
summary:
This paper introduces DynamicRTL, a novel approach for learning representations of digital circuit behavior that incorporates both static structure and dynamic (runtime) execution characteristics. The proposed work has the following contributions:
1. First Comprehensive Dynamic Circuit Dataset
2. Novel Graph Neural Network Architecture (DR-GNN)
3. Extensive experiments show the effectiveness of proposed method

Overall, this work represents a significant step forward in applying machine learning to hardware design and verification, particularly in understanding how circuits behave during execution rather than just their static characteristics. This could have important applications in hardware verification, optimization, and electronic design automation.

strengths:
1. This is the first work to create a comprehensive dataset for dynamic circuit behavior.
2. Extensive experiments demonstrate the effectiveness of proposed method.
3. Novel technical approach combining operation-level CDFG with circuit-aware GNN propagation

weaknesses:
1.  The focuses of proposed work primarily on branch prediction as the main indicator of dynamic behavior. How about other aspects of circuit dynamics such as timing, power consumption, or glitch behavior.
2. It will be better if the authors have some discussion on the limitation of proposed method, especially the effectiveness and efficiency of proposed method for large-scale circuits.

questions:
Please check weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732389725671

thread_type:
official_comment

<thread_object>
creator: area_chair_2kes

<content>
title:
Engage in Discussions Before Nov 26 (AoE)

comment:
Dear Reviewers,

First, let me thank you for your invaluable contributions to the ICLR review process. Your constructive feedback plays a key role in enhancing the quality of submissions.

---

As we approach the final days of the discussion phase (ending **Nov 26, 2024, AoE**), I kindly remind you to:

- Please take a moment to review the authors' responses to your comments. This is an opportunity to clarify any remaining questions, acknowledge misunderstandings, and refine your evaluation.

- If you need further clarification, don't hesitate to post your comments as soon as possible.

- If the authors' responses address your concerns or provide new insights, please consider updating your score to reflect this.

---

Your thoughtful participation during this phase is especially valuable for borderline papers, where additional input can be critical to ensuring a fair decision-making process.

I understand how busy this time of year can be and truly appreciate the time and care you dedicate to this important role. Your efforts make a tangible impact on the success of ICLR.

Thank you once again for your dedication.

Best regards,

Area Chair, ICLR 2025
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces DynamicRTL, a framework for learning dynamic representations of circuits at the RTL level by combining static structural information with dynamic multi-cycle execution. The authors propose a GNN architecture, which operates on operation-level Control Data Flow Graphs (CDFGs) derived from RTL circuits. 

Here are the strengths of the paper:

(a) The dataset is a good contribution to the community (maybe more relevant to dataset tracks), capturing dynamic execution traces and offering potential for future studies in circuit representation learning.

(b) This work addresses the under-explored area of *dynamic* circuit behaviors at the RTL level, providing insights into branch and assertion prediction.

(c) The experiments demonstrate promising results on the proposed tasks, showing transferability of learned representations from branch prediction to assertion prediction.

The reviewers mentioned multiple weaknesses:

(a) The proposed GNN architecture relies on established techniques like Heterogeneous Graph Transformer and GRU, with minimal innovation. The use of operation-level CDFGs is a standard practice in circuit analysis, and while the authors adapt it for dynamic tasks, this seems not to be a significant contribution.

(b) The evaluation focuses only on branch prediction and assertion prediction, which, while relevant, are niche and do not cover broader or more impactful circuit design tasks like timing, power, and area (PPA) estimation.

(c) The baselines are limited to basic GNN models (e.g., GCN, GAT) and an older method (Design2Vec). The paper omits comparisons with more recent state-of-the-art models.

(d) The circuits evaluated are relatively small, with node counts ranging from 10 to 500, leaving scalability to more complex designs untested.

(e) Reviewers raised concerns about hyperparameter tuning, lack of validation datasets, and claims about mitigating over-smoothing in GNNs. The rebuttal responses were insufficiently detailed or dismissive in addressing these points.


While the paper contributes a valuable dataset and targets an interesting problem (dynamic circuit behavior), it lacks the novelty, robustness, and breadth required for acceptance at this stage. The reliance on existing methodologies, narrow task scope, limited scalability, and insufficient comparisons undermine its significance. Additionally, the rebuttal did not satisfactorily address the key concerns raised by reviewers. Therefore, I recommend **Reject** for this work.