Circuit Representation Learning with Masked Gate Modeling and Verilog-AIG Alignment
Authors: Haoyuan WU
Haisheng Zheng
Yuan Pu
Bei Yu
Venue: ICLR 2025 Poster (2025)
Keywords: circuit representation learning
masked graph modeling
large language models
multimodal alignment
Primary area: learning on graphs and other geometries & topologies

Abstract: Understanding the structure and function of circuits is crucial for electronic design automation (EDA). 
Circuits can be formulated as And-Inverter graphs (AIGs), enabling efficient implementation of representation learning through graph neural networks (GNNs).
Masked modeling paradigms have been proven effective in graph representation learning.
However, masking augmentation to original circuits will destroy their logical equivalence, which is unsuitable for circuit representation learning.
Moreover, existing masked modeling paradigms often prioritize structural information at the expense of abstract information such as circuit function.
To address these limitations, we introduce MGVGA, a novel constrained masked modeling paradigm incorporating masked gate modeling (MGM) and Verilog-AIG alignment (VGA).
Specifically, MGM preserves logical equivalence by masking gates in the latent space rather than in the original circuits, subsequently reconstructing the attributes of these masked gates.
Meanwhile, large language models (LLMs) have demonstrated an excellent understanding of the Verilog code functionality.
Building upon this capability, VGA performs masking operations on original circuits and reconstructs masked gates under the constraints of equivalent Verilog codes, enabling GNNs to learn circuit functions from LLMs.
We evaluate MGVGA on various logic synthesis tasks for EDA and show the superior performance of MGVGA compared to previous state-of-the-art methods. 
Our code is available at https://github.com/wuhy68/MGVGA.

Published as a conference paper at ICLR 2025
CIRCUIT REPRESENTATION LEARNING WITH MASKED GATE MODELING AND VERILOG-AIC ALIGNMENT
Haoyuan Wu' Haisheng Z
Yuan Pu
Bei Yu!
'The Chinese University of Hong Kong
2Shangaai Artificial Intelligence Laboratory {hywu24, ypu, byujecse, cuhk. edu. .hk
*ChatEDA Tech
zhenghaishheng@eglab.orr
ABSTRACT
Understanding the structure and function of circuits crucial for electronic design automation (EDA). Circuits be formulated And-Inverter graphs (AIGs), en- abling efficient implementation of representation learning through graph neural networks (GNNs). Masked modeling paradigms have been proven effective in graph representation learning. However, masking augmentation to original cir- cuits will destroy their logical equivalence, which unsuitable for circuit rep- resentation learning. Moreover, existing masked modeling paradigms often pri- oritize structural information at the expense of abstract information such as cir- cuit function. To address these limitations, we introduce MGVGA, novel con- strained masked modeling paradigm incorporating masked gate modeling (MGM) and Verilog-AIG alignment (VGA). Specifically, MGM preserves logical equiva- Ience by masking gates in the latent space rather than the original circuits, sub- sequently reconstructing the attributes of these masked gates. Mcanwhile, large language models (LLMs) have demonstrated an excellent understanding of the Verilog code functionality. Building upon this capability, VGA performs masking operations on original circuits and reconstructs masked gates under the constraints of equivalent Verilog codes, enabling GNNs learn circuit functions from LLMs. We evaluate MGVGA on various logic synthesis tasks for EDA and show the su- perior performance of MGVGA compared previous state-of-the-art methods. Our code is available at https://tthub..cmm//wwy////CCC
INTRODUCTION
In recent years, there has been surge of interest in deep learning for electronic design automation (EDA), which holds great potential for achieving faster design closure and minimizing the need for
extensive human supervision (Wen et al.) 2022 Chen al., 2022 Liang et al.) 2023 Chen et al.)
2023 Wu et al. 2024). Logic synthesis (Hachtel & Somenzi 2005), a vital step in EDA, is process by which an abstract specification of desired circuit behavior is turned into design implementation for logic gates. In the field of logic synthesis, circuits can be formulated graphs (c.g.. Ind-Inverrer graph (AIG) (Mishchenko et al. 2006)), which are well-suited for modeling element connections and topology. Consequently, GNNs (Zhang et al. 2020 Zheng et al. 2024 Chowdhury al. 2022)
have been widely used to learn the characteristics of circuits for various downstream tasks.
The effectiveness of GNNs in logic synthesis has been demonstrated in supervised settings with task- specific labels. However, obtaining labeled data for supervised learning is costly while unlabeled circuit data is available and abundant. This discrepancy makes self-supervised learning suitable for circuit representation learning. Recent works (Wang et al.] 2022;; Shi et al.] (202)) explored leveraging the functional aspects of circuits, such as truth tables and functional equivalence, derive meaningful representations via the self-supervised learning paradigm. These methods efficiently capture the functional behaviors of circuits, which are crucial for many applications.
The structure of a circuit, including its layout, connectivity, gate numbers, and circuit level, plays critical role determining its power, performance, and area (PPA), all of which are key optimiza- tion targets of EDA. Models trained with functional targets often fall short in extracting structural
'These authors contributed equally this work.

---

Published as a conference paper at ICLR 2025
Original AIG
Masked AIG
Circuit Design
Verllog nodute deespe
o
Mask Operation
extandile
oD OD
Logic Synthesis Tool
DO
Possible Reconsiruct AIGs
AIG
Figure Possible reconstruct AIGs for masked AIG. If circuit gates are masked, there are various logic-correct solutions for reconstruction.
Figure 2: Logic equivalence between Verilog code and AIG. For a circuit design, AIG can be translated from Verilog code.
details. Masked modeling paradigms, which have been successfully applied in computer vision (He al. 2022 Bao et al. 2.21), natural language processing (Kenton & Toutanova] 2019), and graph learning (Hou et al. 2023 Li et al. (2033a), offer promising solution to learn detailed structural in- formation. Consequently, we apply the masked modeling paradigm to circuit representation learning to extract a more fine-grained representation of circuit structure.
Challengingly, traditional masked graph modeling paradigm (Hou et al. 2023 Li al. 2023a) can not be applied directly to circuit representation learning which follows strict logical equivalence. In conventional applications, such as social or molecular graphs, masking nodes can provide a unique solution for reconstruction. However, when gates are masked in circuit, their reconstruction will admit various solutions as illustrated Figure [ This is because no matter how we replace gates in the original circuits, logical correctness can still be maintained without necessarily preserving logical equivalence. Consequently, applying traditional masked modeling to circuit representation learning can not guarantee the extraction of circuit-specific features. To address this limitation, we propose a constrained masked modeling paradigm that ensures logical equivalence between the original and reconstructed circuits, thereby enabling effective circuit representation learning.
As mentioned, abstract circuit functions are useful for EDA tasks. However, they are not explic- itly represented in the structural layouts. Such functional attributes are often derived from textual descriptions in hardware description languages (HDLs), which large language models (LLMs) can effectively process. LLMs have demonstrated remarkable performance HDL code generation Pei et al.] ()024); Tsai et al. 2024); Fang et al. (2024); Liu et al. (2023). Consequently, LLMs can guide GNNs in understanding circuit functions. Specifically, the AIG is logically equivalent to the corresponding behavior Verilog code for the same circuit design illustrated in Figure 2 This cquivalence allows for the alignment between Verilog codes and AIGs, facilitating GNNs under- standing of circuit functions.
This study presents MGVGA, a constrained masked modeling paradigm incorporating Masked Gate Modeling (MGM) and Verilog-AIG Aligmment (VGA) for eircuit representation learning. Firstly, we introduce MGM to mask gates in the latent space instead of masking in the original circuits. This approach can use unmasked gate representations constraints maintain logical equivalence during masked gate reconstruction. as these representations already capture features from masked gates. However, GNNs trained with MGM alone focus primarily on structural relationships within circuits, potentially overlooking function features. To address this, we design VGA to leverage LLMSs expertise in Verilog code to transfer functional knowledge to GNNs by aligning AIGs with equivalent Verilog codes. Specifically, VGA performs masking operation on the original circuits and reconstructs masked gates under the constraint of corresponding Verilog codes. Although this operation will destroy the logical equivalence of the original circuits when cooperating with the traditional masked modeling strategy, we can still utilize equivalent Verilog codes as constraints to ensure the logic equivalence during the reconstruction process.
In summary, our main contributions are as follows:
Propose masked gate modeling (MGM) for circuit representation learning, enabling GNNs to extract eircuit representations with fine-grained structural information.
2

---

Published as a conference paper at ICLR 2025
Develop the Verilog-AIG alignment (VGA), which employs LLMs teachers to guide GNNs to extract circuit representations with abstract circuit functions through equivalent AIGs and Verilog codes alignment.
Conduct extensive evaluations and show superior performance on various logic synthesis tasks including quality of result (QoR) and logic equivalence identification compared to previous state-of-the-art (SOTA) methods.
RELATED WORK
AIG-Formatted Circuit Representations for Logic Synthesis. An AIG is a directed acyclic graph (DAG) utilized for representing circuits (Brummayer & Biere 2006), which is composed of AND gate, NOT gate, and terminal nodes that serve as primary inputs (PIs) and primary outputs (POs). Considering its simplicity, AlG-formatted circuits are widely used to perform circuit representation learning via GNNs. For example. Zhang et al. (2020) employs a GNN model to predict circuit's power consumption, and Zheng et al. (2024) proposes a customized GNN predict the delay of cir- cuits accurately. Moreover, Wang et al. (2022) and Shi et al. (2023) perform self-supervised learn- ing for extracting general AlG-formatted circuit representation.
this study, we convert circuits to AIGs to perform general circuit representation learning via a self-supervised learning paradigm.
Masked Graph Autoencoder: Masked autoencoders (He et al. 2022 Bao et al. [022] are grounded in the masked modeling learning paradigm, which involves masking portion of the input signals and predicting the obseured content. Graph autoencoders (Hou et al. 2023] Li et al. 2023a) employ an autoencoder architecture to encode nodes into latent representations and reconstruct the graph from these embeddings. Integrating the strengths of the above methods, the masked graph autoen- coder has been introduced to advance representation learning for graph-structured data. The masked graph autoencoder randomly masks a subset of graph nodes and reconstructs them using information from the unmasked nodes and their structural connections. This approach compels the encoder to decipher the underlying relational patterns within the graph, thereby generating robust and informa- tive node representations. In this paper, we apply the constrained masked modeling paradigm for general circuit representation learning through the masked graph autoencoder.
LLM-based Embedding Models. LLMs, structured as decoder-only architectures. inherently face challenges in effectively encoding bidirectional context, which can impede their capacity to gen- erate comprehensive and discriminative embeddings. Consequently, researchers apply contrastive learning for representation learning of LLMs to leverage the natural language comprehension capa- bilities of LLMs for embedding-related tasks (Muennighoff et al. 2024 [BehnamGhader et al. a 22024] Li et al. 2023b Lee al., 2024). This kind of strategy can help LLMs extract better representation through bidirectional attention mechanisms without hurting their abilities. In this study, we utilize LLMs to extract Verilog codes embedding with comprehensive understanding of circuit function, serving as constraints of the reconstruction process of VGA.
METHODOLOGY
Due to logical equivalence issues, traditional mask graph modeling techniques are inadequate for circuit representation learning. Additionally, GNNs have inherent limitations in extracting abstract circuit functions. To address these challenges, we propose MGVGA, a novel constrained masked modeling strategy incorporating masked gate modeling (MGM) and Verilog-AIG alignment (VGA) for enhanced circuit representation learning. AIGs have gained widespread adoption in circuit rep- resentation learning. Consequently, we convert eircuits to AIGs to implement our MGVGA. In the following subsections, we will elucidate the details of MGM (Section 3.2) and VGA (Section 3.3) utilizing the AIG autoencoder. Notably, we also provide a detailed illustration of how we preserve the logic equivalence when performing MGVGA in Appendix A.I.I
3.1 AIG AUTOENCODER
Let 9 = (V,A)) represent an AIG, where denotes the set of N nodes, d2 E V, categorized into four types: PI, PO, AND, and NOT gates, each labeled by o; C, (1,2, 11,2,3,4 4). The adjacency matrix
3

---

Published as a conference paper at ICLR 2025
[MASK]
AND Gate
NOT Gate
Share Weights
GNN Encoder
Masked Gabes
MOMVGA
AIG
Masking
GNN Enooder
Constrain Block
VGA
Galte Prediction
Lpp
Decader
Gale-LLve
Prediction
Daurrs
10
Masked
Verlog
Indegres
Outdegree
Venlogg AlG Alignment
Figure 3: Overview of the MGVGA for circuit representation including masked gate modeling and Verilog- AIG alignment. For both MGM and VGA, the AIG reconstruction is implemented by gate type prediction and gate-level degree prediction from reconstructed representation.
A E (0, 1) N shows the connectivity between nodes, where As, represents an existing edge from uy to uj. A delineates the structure and the types of connections within g.
For an AIG autoencoder, a GNN encoder, de- noted by gE, encodes G into latent space rep- resentation X Nxd where d represents the dimension of this representation. The encoding process of an AIG can be formulated as:
Bidrectional LLM
Constraint AlG Representation
Adapriu
Cross Altention Block
Xy
X = AV,
(1)
rodile in desigr
Concurrently, a GNN decoder, gn. endeavors to reconstruct the AIG 9 from according to:
outaut
andecduul
Masked AIG.
vyga
Vgga
(X = 9
(2)
Verilag Code
Representation
where 9 denotes the reconstructed graph, po- tentially encompassing both node features and structure. The primary objective of the AIG autoencoder is optimize the encoder gE produce an effective representation X that facilitates the accurate reconstruction the original G. Notably, fol- lowing previous masked modeling paradigms (Hou et al. [2222] 2023), we do not mask the adjacent matrices during the entire process and the detailed reasons will be explained in Appendix A.
Figure 4: The constraint block for VGA.
3.2
MASKED GATE MODELING
The idea of the masked autoencoder has been applied successfully graph self-supervised learn- ing. As an extension denoising autoencoders, masked graph autoencoder (Hou et al.) 2023 Li
al. (2023a) selectively obscure portions of the graph, such as node features or edges, through masking operation, and learn to predict the obscured content. In traditional masked graph autoen- coders, nodes are typically masked directly in the input graph before being processed through the autoencoder framework. However, this approach presents significant challenges when applied to structurally constrained graphs like AIGs. which follow strict logical rules. Utilizing straightfor- ward random masking technique will lead to reconstructed logic expressions that diverge from their original forms, which can be tolerated.
To address this, we propose masked gate modeling, where the AIG is initially processed unmasked through the encoder to capture its latent representation. Rather than masking nodes at the original AIG, the masking operation is applied to the encoded representation in the latent space. During the masked modeling process, the encoder can retain the complete logical structure of the AIG before masking. This approach allows the representations of unmasked gates to reconstructing the attributes of masked gates, as the latent representations of unmasked gates have already aggregated some features from masked gates.
constraints for
4

---

Published as a conference paper at ICLR 2025
Formally, as depicted in Figure we uniformly sample subset of gates Vingm V without replace- ment and replace the remaining nodes with the mask token [MASK], which can be represented by a learnable vector E IR. Consequently, the masked node representation X, Xmgm for each node 0 is given by:
Xi, if 2s Vmgmi
x;
m,
if og & Vmgm-
The X then fed into the decoder gD to reconstruct the following Equation 2 As illustrated in Figure 3 the decoder maintains the connectivity of each node and generates the reconstructed node representation X Xmgm E Rg-xd for G reconstruction (Section 3.4).
3.3 VERILOG-AIG ALIGNMENT
Although GNNs enhanced with MGM excel at extracting structural information from circuits, they often struggle to capture abstract circuit functions that are not explicitly represented in structural layouts. Meanwhile, Verilog codes contain substantial semantic information, including high-level abstract concepts and functional logic in circuit designs. Recent studies (Lu et al.) [024] Pei et al.) 2024) have begun leveraging LLMs to analyze Verilog codes, highlighting the potential for distilling circuit function knowledge from LLMs to GNNs. LLMs can serve as excellent teachers, guiding GNNs understanding circuit functions through the alignment process of equivalent Verilog codes and AIGs. Meanwhile, illustrated in Figure AIGs are translated from Verilog codes via logic synthesis tools. Consequently, there exist equivalent behavior-leve Verilog codes for AIGs. Based on equivalent Verilog-AIG pairs, operations on original AIGs. Subsequently, the masked gates are reconstructed under the constraints of equivalent Verilog codes. This equivalence allows for the reconstruction of masked AIGS under the supervision of Verilog code, facilitating GNNs' understanding of circuit functions.
perform Verilog-AIG alignment, which conducts masking
Similar to MGM, uniformly sample subset of gates Vuga without replacement and replace the node types of remaining nodes with Cm, which represents these nodes are masked in the original AIG. Consequently, for 2/ E V of the masked AIG, the node type o can be defined as:
if Vigai
C
(4)
uf
Vuga-
The masked AIG & (V.A) is fed into the encoder gE to generate the encoded masked AIG representation Xyya following Equation (1)
As mentioned earlier, the reconstruction of G from Xvgn must be constrained by equivalent Verilog code to ensure strict logical equivalence. Consequently, we design constraint block inspired by Jaegle et al. (2021) and Lee et al. (2024) as illustrated in Figure 4 Specifically, we perform adaptive pooling on token embeddings of Verilog code generated by LLMs extract X E TR M the representation of the equivalent Verilog code. We then feed the masked AIG representation X and Verilog code representation Xv into a cross-attention block C. to perform alignment between the masked AIG and Verilog code, with Xvya being projected to query Q E IR and Xv being projected to key K E: RMxx xd and value V E R xd Specifically, we selected M = 16 after carefully balancing computational cost and performance. The output of the cross attention block (Vaswani 2017 is the constrained AIG representation X E IR Nxd, which can be calculated as follows:
QKT
X X C(Xvgn Xv) Softmax
V.
5
Vd.
After aligning the logically equivalent Verilog code and masked AIG, X from the abstract circuit function extracted by LLMs. Then, illustrated Figure 3 we obtain the reconstructed circuit representation X.ga from Xiga via gn following Equation (2) while preserving logical equivalence. Subsequently, X will be utilized for G reconstruction, detailed in Section 3.4
vga incorporates information
3.4 AIG RECONSTRUCTION
As illustrated in Figure 3 given the reconstructed node representations from MGM or VGA, predict the attributes of masked nodes. First, we predict the types of each masked node, categorizing them AND gate, NOT gate, PI, POs. Next, focus the specific attributes of the nodes
5

---

Published as a conference paper at ICLR 2025
themselves. Specifically, AND gates have two inputs (which may be identical), while NOT gates have only one input. Moreover, PIs have no inputs, and POs have no outputs. Consequently, we pre- dict the degree of each masked node to help GNNs learn the attributes of each gate more effectively. Moreover, gate-level degree prediction aids GNNs in capturing the connectivity between gates and the overall structures of AIGs, as illustrated in Figure For elarity, we unify the reconstructed node representations from MGM (Xmgm Xd) and VGA (Xvga Wxd into a single notation X.
Gate Type Prediction. For gate type prediction, X is transformed by mapping function fyyee R + IC into a categorical probability distribution over C' classes. This leads to the formation of the overall probability distribution matrix Ž E R.VxC, where each element Z,, represents the softmax-estimated probability that node v; belongs to class cj. Importantly, the gate type recon- struction loss calculated only for the Nm masked nodes:
Lype
ΣE
Σ E
l. log Zi,-
(6)
N
where Y c {0, NxxC and Yú is binary indicator that equals 1 if the node vy belongs class ej and O otherwise.
Gate-Level Degree Prediction. The gate-level degree prediction involves forecasting the in-degree and out-degree of each masked gate within the AIG. Formally, given the reconstructed node repre- sentations X. in-degree labels D E ER, and out-degree labels D+ E RR,, we utilize mean squared error the loss function for degree regression tasks. The degree reconstruction loss, calculated only for the Nm masked nodes, is defined as:
Liegrrse
- (D, - (X:))²)
Nm
where fin : Rd and fout R2 E serve mapping functions for predicting gate-level degrees. This task allows GNNs to infer the connectivity between nodes, providing insights into the interaction patterns for understanding the attributes of each gate and the organization of the circuits. As illustrated in Figure the AIG reconstruction is implemented by gate type prediction and gate- level degree prediction from reconstructed representation for both MGM and VGA. Consequently, the AIG reconstruction loss for MGM and VGA can be defined as:
Lmgm/spe /vga Laype + Ldegree
3.5 CONSTRAINED MASKED MODELING: MGVGA
Building upon the methodologies of MGM and VGA based on the AIG autoencoder, we propose a novel constrained masked modeling paradigm, MGVGA, to perform general circuit representa- tion learning. This paradigm synthesizes these strategies to develop GNNs that effectively capture diverse and intricate features circuits. Formally, the loss function for MGVGA can be defined as:
Emgyaa = Lmgm + Cuga.
This integration enables the GNNs learn concurrently from fine-grained structural information and abstract circuit function features, optimizing a unified representation that facilitates a wide range of logic synthesis tasks such as classification, regression, and complex reasoning on circuits.
EXPERIMENTS
4.1 DATA PREPARATION
AIG Collection For MGM. We obtain 27 circuit designs from five circuit benchmarks as our train-
ing dataset: MIT LL Labs CEP (Chetwynd et al., 2019). ITC'99 (Davidson, 1999). IWLS S'05 (AI- brecht 2005), EPFL (Amarú et 2015), ) and OpenCore (Takeda 2008). The resulting AIG dataset comprises 8 810000 AIGs and 40500 synthesis labels across various optimization sequences and cir- cuit designs. We provide more details of the AIG collection in Appendix A.2.1

---

Published as a conference paper at ICLR 2025
Verilog-AIG Pairs Collection For VGA. In this phase, source Verilog codes (Thakur et al. 2023 Liu et al., 2023) are selected and subjected to logic synthesis using Yosys (Wolf et al. (013)) and then they converted into AIG format. This process yields 64826 Verilog-AIG pairs, which are utilized for VGA illustrated in Section 3.3
AIG Preprocessing. As mentioned previously, we convert circuits to AIGS implement MGVGA. The node type of AIG can be categorized into PI, PO, AND, and NOT gates. Given that the number of Pls and POs typically minimal, the primary emphasis in masked modeling lies in the accurate reconstruction of AND and NOT gates. It is worth noting that the NOT gate is the only single-imput logic gate. Our concern is that the model could potentially leverage the disparity in in-degrees of gates as a shortcut, thereby simplifying the reconstruction task without learning the useful circuit representations. Consequently, we introduce single-input AND gates during the training phase as illustrated in Figures [to 3 which have two identical inputs. The input and output of the single-input AND gate are identical, making this augmentation simply adaptable to circuits featuring various logic gates (NAND, XOR, OR, etc.). Our experiments indicate that GNNs struggle to precisely capture degree information during the reconstruction process. Consequently, we employ this augmentation method as trick in our training process, treating it as equivalent augmentation for AIGs to avoid overfitting and possible leakage. Notably, this augmentation not utilized during the evaluation phase as shown in Figure
Evaluation Dataset Collection. As for the evaluation dataset, we select 10 circuit designs external to the training dataset from opensource benchmark (Chowdhury et al., 2021 Amará et al. 2015 OpenRISC 2009; YosysHQ 2020 (Asanovic 2016). the details of which are illustrated in Table Additionally, we will provide more details on benchmark selection in Appendix A.2.4
4.2 IMPLEMENTATION DETAILS
Training Process of MGVGA. For the circuit representation learning utilizing our MGVGA paradigm, we utilize DeepGCN (Li et al. 2019; 2020) as the GNN encoder and decoder. As for the LLM, we utilize gte- -wen---w--instttee
(Li et al. 2023b), trained with bidirectional attention mechanisms based on Qwen2-7B (Yang et al. 2024). which has a comprehensive understanding of abstract circuit function described in Verilog codes (Liu al. 2023 Pei et al., 2024 Tsai et al.]
2024 Fang et al. 2024). The training process employs a linear learning rate schedule with the Adam optimizer set at learning rate of 10-3, weight decay of 0.01, and a batch size of 512. The model is fine-tuned for 3 epochs on 8 GPUs with 80G memory each. Additionally, provide more details about the model settings in Appendix A.2.3
Baseline Selection. As for the baseline selection, select DeepGate2 (hi al. 2023 baseline due to its similar scope and SOTA performance in general circuit representation learning. Additionally, we provide more details of baseline selection in Appendix AA.2.2
(..222
Evaluation. To validate the efficacy of our MGVGA, we conduct evaluations across two distinct logic synthesis tasks including the Quality of Results (QoR) Prediction and Logic Equivalence Iden- tification tasks. During the evaluation process, we utilize the GNN encoder trained with MGVGA to extract the AIG representation without extracting Verilog code representations. Moreover, tract the AIG representation directly without fine-tuning DeepGate2 and MGVGA for downstream tasks. Notably, QoR prediction aims to assess the ability to extract structural information, whereas logic equivalence identification is designed to evaluate the capability of extracting abstract function information. Moreover, besides identifying logic equivalence directly, we conduct experiments on the boolean satisfiability solving (SAT) task in Appendix [A.3.2] SAT solving requires rough logic equivalence checking to be strictly validated later. Additionally, we provide more details about the model settings in Appendix A.2.3
4.3 QOR PREDICTION
For QoR prediction tasks, we estimate the number of optimized gates for the circuit designs fol- lowing logic synthesis optimization via ABC (Brayton & Mishchenko 2010). As illustrated in Figure 5(a), utilize GNN encoder extract circuit embeddings exclusively from AIGs. These embeddings are extracted to train the QoR prediction model with the datasets described in Sec- We evaluate the performance across ten circuit designs as illustrated in Table [ with
tion 4.1
7

---

Published as a conference paper at ICLR 2025
Qoff Prediction
Logio Equivalence identitoation
Regression
Gasine Similrity
Sequencee Embedcing
Grzui Embeddiigg
Graii Embeddiing
Croat Embedding
Embedding
Synthesis Embeading Layar
MGVGA
MGVGA
Optimization Sequeecee
Op
DO
De
AG
A13
AIS
AKS
Figure 5: Application of MGVGA in QoR prediction and logic equivalence identification.
Table 1: Performance of DeepGate2 and MGVGA QoR prediction.
DeepGate2
MGVGA (Ours)
Design
#P
PO
# Gates
NDCG@A
Top-4S:
Commonality
NDCG@
Top-AS%
Commonality
km3
km
km3
km3
k=10
km3
km3
km3
k=5
k=10
11
2784
0.331
0.395
0.244
0.227
0.280
0.444
0,500
0.222
0.213
0.320
apexl
45
45
2661
0.645
0.643
0.222
0.133
0.413
0.706
0.716
0.311
0.400
0.513
128
101698
.0.063
0.029
-0.050
k2
45
45
4075
-0.060
0.040
0.022
0.040
0.080
0.902
0.873
0.267
0.320
0.400
i10
257
224
3618
-0.133
40,000
0.000
0.000
0.027
0.620
0.607
0.289
0.307
0.353
mainpla
9441
0.674
0.629
0.267
0.293
0.360
0.594
0,598
0.200
0.187
oe1200,cpu
2343
2072
56570
0.498
0.485
0.178
0.267
0.407
0.617
0.613
0,222
0,253
0.367
picurv32
1601
0.457
Rocket
4413
4187
96507
0.578
0.543
0.111
0.186
0.300
0.557
0.607
0.355
0.413
0.467
sgri
128
64
40920
0.304
0.153
0.000
0.027
0.080
0.577
0.401
0.000
0.040
0,080
Average
0.334
0.324
0.116
0.157
0.226
0.540
0.537
0.193
0.231
0.301
each circuit undergoing synthesis through 1500 optimization sequences, each containing 20 steps. Notably, we detail the evaluation process and evaluation metries of QoR prediction task in Ap- pendix [A.2.5] Specifically, we utilize two evaluation metrics including NDCG@K for k = 3, 5 and Top-k% Commonality for k 3, ; 10 in our experiments.
Table Iillutrates detailed comparison berween MGVGA and DeepGate2 in QoR prediction, using the post-synthesis number of gates. MGVGA excels in NDCG@N k for k 3, 5 and achieves higher percentages in Top- Commonality for & = 3, 5, 10 across various designs. When considering the average performance, MGVGA notably surpasses DeepGate2 with an NDCG@3 3 of 0.540 compared to 0.334, and a Top-10% Commonality score of 0.301 against 0.226. This demonstrates MGVGA's superior capability in extracting structure information of circuits, which facilitates recommending optimal optimization sequences. Collectively, these results quantitatively validate the significant ad- vancement of MGVGA over Deep2ate2 in logic circuit optimization tasks, particularly in accurately and efficiently predicting superior gate configurations enhance overall design quality.
4.4 LOGIC EQUIVALENCE IDENTIFICATION
Each design in Table Taale_andeed unddeggess optimization to generate various graph expressions while ensuring functionality equivalence. The Cone, specifying the PIs and PO constructs, also denotes function- ality equivalence. To evaluate the GNNs ability to identify circuit function, we derive logically equivalent gates by isolating the Cone among the designs in Table 2 within AIGS using the cone command within logic synthesis tool ABC (Brayton & Mishchenko 2010). Our dataset consists of 10000 pairs of logic gates to test the identification of logic equivalence. As illustrated in Figure GNNs deem pairs of logic gates as equivalent if the cosine similarity between their embeddings ex- ceeds a predefined threshold during the evaluation process. The predefined threshold optimized based on the receiver operating characteristic (ROC) curve. In our experiments, assess the GNNs' performance using precision, recall, Fl -core,,,, and the area under the ROC curve (AUC).
As illustrated in Table 2 our proposed MGVGA has shown significant superiority over the es- tablished DeepGate2. The comprehensive analysis reveals that MGVGA outperforms DeepGate2, achieving an F1-score of 0.470 compared to 0.424, and an AUC of 0.862 versus 0.804. Moreover, also provide more experiment results for the comparison between DeepGate3 and MGVGA using small designs in Appendix A.3.1 These results underscore the consistency of MGVGA and its
8

---

Published as a conference paper at ICLR 2025
Table 2: Performance of DeepGate2 and MGVGA on logic equivalence identification.
DeepGate2
MGVGA (Ours)
Design
Precision
Recall
Fl Score
AUC
Precision
Recall
Fl Score
AUC
0.199
0.930
0.327
0.813
0.274
0.715
0.396
0.817
apex
133
0.680
0.223
0.601
0.273
0.710
0.394
0,826
0.980
0.725
k2
0.171
0.720
0.276
0.695
0.336
0.920
0.492
0.919
0.414
0.940
0.575
0.918
0.699
0.950
0.985
mainpla
0.178
0.790
0.290
0.732
0.167
0.900
0.281
0.746
orl ee1200_cpu
0.451
0.790
0.575
0.823
0.356
0.950
0.518
0.929
picorv32
0.448
0.870
0.592
0,918
0.440
0.960
0.604
0,941
Rocket
0.346
0.930
0.504
0.892
0.388
1.000
0.559
0.952
sqrt
0.189
0.740
0.302
0.721
0.199
0.720
0.312
0.770
Average
0.295
0.841
0,424
0.804
0.336
0.848
0,470
0.862
Table 3: Performance of MGVGA on QoR prediction and logic equivalence identification with different masking ratios of MGM and VGA.
Masking Ratio
QoR Prediction
Logic Equivalence Identification
MGM
VGA
k=3
NDCG@1 f
K=5
Top-I-k Commonality k=5
i=33
k=10
Precision
Recall
Fl-Score
AUC
0.3
0.3
0.517
0.505
0.158
0.199
0.272
0.300
0.844
0.430
0.846
0.3
0.5
0,540
0.537
0.193
0.231
0.301
0.336
0.848
0.470
0.862
0.498
0.178
0.223
0.299
0.316
0.823
0.5
0.3
0.439
0.445
0.149
0.187
0.271
0.274
0.821
0.402
0,821
0.5
0,438
0,470
0.169
0.204
0.288
0.331
0.829
0,450
0.836
0.5
0.7
0.385
0.415
0.140
0.168
0.247
0.304
0.833
0.433
0.817
0.7
0.347
0.367
0.127
0.160
0.242
0.292
0.871
0.421
0,828
0.7
0.5
0.400
0.366
0.111
0.175
0.228
0.313
0.823
0.433
0.852
0.7
0.7
0.366
0.359
0.138
0.169
0.226
0.305
0.794
0.425
0.822
perior ability accurately determine functionally equivalent circuits, highlighting its effectiveness in extracting abstract circuit function features.
4.5 ANALYSIS OF MASKING RATIO
This section analyzes the impact of masking ratios for both MGM and VGA. Table 3 indicates that the optimal masking ratio for MGM is 0.3, while the optimal masking ratio for VGA is 0.5. At an MGM masking ratio of 0.3, the MGVGA method demonstrates notable performance. Meanwhile, there is consistent improvement in both QoR prediction and logic equivalence identification tasks when the VGA masking ratio increases from 0.3 to 0.5.
The study reveals that higher MGM masking ratios negatively affect QoR performance, suggesting that excessive masking impedes the GNNs' ability to learn information effectively. Specifically, excessively high masking ratios (0.5 and 0.7) significantly reduce the performance of MGVGA in the QoR prediction task, which reflects the capability in structural circuit information extraction. As for VGA, a relatively higher masking ratio (0.5) generally yields better performance for the logic equivalence identification task. An excessively high masking ratio (0.7) degrades the performance of extracting circuit structural and functional features. These findings align with our intuitive ex- pectations. MGM directly enables GNNs recover complete structural information from masked latent space. GNNs can not learn effective information from unmasked gates if the masking ratio is too high. In the VGA task, introducing Verilog codes as constraints for circuit restoration allows for a relatively high masking ratio, as Verilog codes contain rich circuit information.
4.6 EFFECTIVENESS OF CONSTRAINT MASKED MODELING
To assess the effecctiveness of our MGM and VGA approaches, we conduct an ablation study QoR prediction and logic equivalence identification tasks. We use the original masked modeling strategy (Hou et al. 2023) MGM methods, based on the optimal performance observed in Table
baseline, employing masking ratio of 0.3 for both the original and
As shown in Table 4 the original masked modeling strategy yielded poor performance in both tasks. This outcome aligns with our previous assertion that masking in the original circuits disrupts their logical equivalence, thereby preventing the method from learning effective eircuit features. In con-
9

---

Published as a conference paper at ICLR 2025
Table 4: Ablation study on constraint masked modeling paradigm, including MGM and VGA.
Mask Stratogy
QoR Prediction
Logic Equivalence Identification
MGM
VGA
NDOG@ET
k=3
k=5
Top-A/%
K=3
Commorality
K=5
k=10
Precision
Recall
FI-Score
ATCC
0.153
0.207
0.107
0.159
0.255
0.264
0.793
0.382
0.790
0.338
0.368
0.135
0.197
0.289
0.305
0.850
0.435
0.833
0.540
0.537
0.193
0.231
0.301
0.336
0.848
0.862
Table 5: Generalization on various GNNs, including GraphSAGE and graph transformer.
QoR Prediction
Logic Equivalence Identification
GNNs
NDCG@k 1
kw3
kw5
Top-k%
ke3
Commonality 1
kw5
k=10
Precision
Recall
FI-Score
AUC
DeepGate2
0.334
0.324
0.116
0.157
0.226
0.295
0.841
0.424
0.804
GraphSAGE
0.469
0.479
0.153
0.224
0.314
0.329
0.811
0.455
0.841
Graph Trancformer
0.452
0.470
0.134
0.212
0.311
0.324
0.789
0.450
0.831
DeepGCN (Ours)
0.540
0.537
0.193
0.231
0.301
0.336
0.848
0.470
0.862
trast, both our MGM and VGA approaches demonstrated significant improvements in the two tasks, underscoring their effectiveness in enhancing GNNs' capacity to extract fine-grained structural in- formation and abstract functional data. Furthermore, VGA not only facilitates GNNs' extraction of circuit function features but also enhances their ability recognize circuit structure during the re- construction process. This improvement occurs under the constraint of logically equivalent Verilog codes, highlighting the versatility of our VGA approach.
4.7 GENERALIZATION ON VARIOUS GNNs
To evaluate the generalization capability of our MGVGA, we perform circuit representation learn- ing using various traditional GNNs, including GraphSAGE (Hamilton al., 2017) and graph trans- former (Shi et al. (021I. Similarly, based on the optimal performance observed in Table [ where MGVGA achieves the best results with the MGM masking ratio of 0.3 and the VGA masking ratio of 0.5, we apply these same masking ratios to the constrained masked modeling of other GNNs.
As shown in Table all GNNs trained with MGVGA exhibited significant improvements compared to the baseline DeepGate2 model. These results demonstrate the exceptional generalization ability of our proposed methods across different GNN architectures. This consistent performance enhance- ment across various models underscores the robustness and versatility of our MGVGA paradigm in extracting better circuit representation via circuit representation learning.
5 CONCLUSION
In conclusion, this study introduces a novel constrained mask modeling paradigm, MGVGA, for circuit representation learning. This method integrates MGM and VGA extract fine- fine-grained struc- tural information and abstract functions circuits. MGM operates by masking gates in the latent space rather than in the original circuits, subsequently reconstructing the attributes of these masked gates. This approach preserves the logical equivalence of circuits, overcoming the limitations of traditional masked gate modeling strategies. Moreover, we developed VGA, which performs mask- ing operations on the original circuits and reconstructs them under the constraints of equivalent Verilog codes. This enables GNNs learn circuit functions from LLMs. Our comprehensive eval- uations demonstrate that MGVGA performs better than previous SOTA methods in QoR prediction and logic equivalent identification tasks. This represents a significant advancement in applying the constrained masked modeling paradigm to general circuit representation learning.
ACKNOWLEDGEMENTS
This work is partially supported by The Research Grants Council of Hong Kong SAR (No. RFS2425-4S02, No. CUHK1421 1824, No. CUHK14210723), AI Chip Center for Emerging Smart Systems (ACCESS), Hong Kong SAR, and Shanghai Artificial Intelligence Laboratory.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Christoph Albrecht. IWLS 2005 benchmarks. JEEE/ACM International Workshop on Logic Synthe- sis, 2005.
Luca Amarú, Pierre-Emmanuel Gaillardon, and Giovanni De Micheli. The EPFL combinational benchmark suite. In IEEE/ACM International Workshop Logic Synthesis, 2015.
Krste others Asanovic. The Rocket Chip Generator. Berkeley, Tech. Rep. UCB/EECS-2016-17 17, 2016.
EECS Department, University of California,
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEiT: BERT Pre-Training of Image Trans- formers. In International Conference on Learning Representations (ICLR), 2021.
Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapa- dos, and Siva Reddy. LLM2Vee: Large Language Models Are Secretly Powerful Text Encoders. arXiv preprint, 2024.
Robert Brayton and Alan Mishchenko. ABC: An academic industrial-strength verification tool. In International Conference on Computer-Aided Verification (CAV). pp. 24 40, 2010.
Robert Brummayer and Armin Biere. Local Two-Level And-Inverter Graph Minimization without Blowup. Proc. MEMICS, 6:32-38, 2006.
Hao Chen, Kai-Chieh Hsu, Walker J Turner, Po-Hsuan Wei, Keren Zhu, David Z Pan, and Haoxing Ren. Reinforcement Learning Guided Detailed Routing for Custom Circuits. In ACM Interna- tional Symposium on Physical Design (ISPD), pp. 26-34, 2023.
Jingsong Chen, Jian Kuang, Guowei Zhao, Dennis J-H Huang, and Evangeline FY Young. PROS 2.0: A plug-in for routability optimization and routed wirelength estimation using deep learning. IEEE Transactions on Computer- Design of Integrated Circuits and Systems (TCAD), 42 (1):164-177,) 2022.
Brendon Chetwynd, Kevin Bush, and Kyle Ingols. Common Evaluation Platform. https:/ gi Fithub.. com/ miit-11/CEP git, 2019.
Animesh Basak Chowdhury, Benjamin Tan, Ramesh Karri, and Siddharth Garg. OpenABC-D: A Large-Scale Dataset For Machine Learning Guided Integrated Circuit Synthesis. arXiv preprint, 2021.
Animesh Basak Chowdhury, Benjamin Tan, Ryan Carey, Tushit Jain, Ramesh Karri, and Sid- dharth Garg. Bulls-Eye: Active few-shot learning guided logic synthesis. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), 42(8):2580-2500 2022.
Scott Davidson. Characteristics of the ITC'99 benchmark circuits. In IEEE International Test Synthesis Workshop (ITSW), 1999.
Wenji Fang, Mengming Li, Min Li, Zhiyuan Yan, Shang Liu, Hongee Zhang, and Zhiyao Xie. AssertLMM: Generating and Evaluating Hardware Verification Assertions from Design Specifica- tions via Multi-LLMs. arXiv preprint, 2024.
Winston Haaswijk, Mathias Soeken, Alan Mishchenko, and Giovanni De Micheli. Sat-hased exact synthesis: Encodings, topology families, and parallelism. IEEE Transactions Computer-Aided Design of Integrated Circuits and Systems (TCAD), 2019.
Gary D Hachtel and Fabio Somenzi. Logic synthesis and verification algorithms. Springer Science & Business Media, 2005.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Anmual Conference on Neural Information Processing Systems (NIPS), 2017.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked Autoencoders Are Scalable Vision Learners. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16000-16009, 2022.

---

Published as a conference paper at ICLR 2025
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. InACM SIGKDD Conference on Knowl- edge Discovery and Data Mining (KDD, pp. 594-604, 2022.
Zhenyu Hou, Yufei He, Yukuo Cen, Xiao Liu, Yuxiao Dong, Evgeny Kharlamov, and Jie Tang. GraphMAE2: A Decoding Enhanced Masked Self- SelfrSsue Graph Learner. In ACM Web Con- ference (WWW), pp- 737-746, 2023.
Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, et al. Perceiver io: A general architecture for structured inputs & outputs. arXiv preprint urXiv:210..14990 2021.
Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Transactions Information Systems (TOIS), 20(4):422 446, 2002.
Kalervo Järvelin and Jaana Kekäläinen. IR evaluation methods for retrieving highly relevant docu- ments. In ACM SIGIR Forum, volume 51, pp. 243-250, 2017.
Jacob Devlin Ming-War Wei Chang Kenton and Lee Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pp. 4171-4186, 2019.
Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catan- zaro, and Wei Ping. NV-Embed: Improved Techniques for Training LLMs as Generalist Embed- ding Models. arXiv preprint arXiv:2405. / 17428, 2024.
Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. DeepGCNs: Can GCNs go as deep as CNNs? In IEEE International Conference on Computer Vision (ICCV), pp. 9267-9276, 2019.
Guohao Li, Chenxin Xiong, Ali Thabet, and Bernard Ghanem. DeeperGCN: All you need train deeper GCNs. arXiv preprint arXiv:2006.07733 2020.
Jintang Li, Ruofan WU, Wangbin Sun, Liang Chen, Sheng Tian, Liang Zhu, Changhua Meng, Zibin Zheng, and Weiqiang Wang. What's Behind the Mask: Understanding Masked Graph Model- ing for Graph Autoencoders. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 1268-1279, 2023a.
Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281. 2023b.
Rongjian Liang, Siddhartha Nath, Anand Rajaram, Jiang Hu, and Haoxing Ren. BufFormer: A Generative ML Framework for Scalable Buffering. In IEEE/ACM Asia and South Pacific Design Automation Conference (ASPDAC), pp. 264-270, 2023.
Shang Liu, Wenji Fang, Yao Lu, Qijun Zhang, Hongce Zhang, and Zhiyao Xie. RTLCoder: Outper- forming GPT-3.5 in design RTL generation with our open-source dataset and lightweight solution. arXiv preprint, 2023.
Yao Lu, Shang Liu, Qijun Zhang, and Zhiyao Xie. RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model. In IEEE/ACM Asia and South Pacific Design Automation Conference (ASPDAC), 2024.
Alan Mishchenko, Satrajit Chatterjee, and Robert Brayton. DAG-aware AIG rewriting a fresh look at combinational logic synthesis. In ACM/IEEE Design Automation Conference (DAC), pp. 532- 535, 2006.
Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. Generative Representational Instruction Tuning. arXiv preprint, 2024.
OpenRISC. OpenRISC OR 1200
http:://gghhaa.c////arrrrr/////p 2009.
12

---

Published as a conference paper at ICLR 2025
Zehua Pei, Hui-Ling Zhen, Mingxuan Yuan, Yu Huang, and Bei Yu. BetterV: Controlled Verilog Generation with Discriminative Guidance. arXiv preprint, 2024.
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification. In International Joint Conference on Artificial Intelligence (IJCAI), 2021.
Zhengyuan Shi, Hongyang Pan, Sadaf Khan, Min Li, Yi Liu, Junhua Huang, Hui-Ling Zhen, Mingx- uan Yuan, Zhufei Chu, and Qiang Xu. DeepGate2: Functionality-awe aware circuit representation learning. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 1-9, 2023.
Zhengyuan Shi, Ziyang Zheng, Sadaf Khan, Jianyuan Zhong. Min Li, and Qiang Xu. Deepgate3: Towards scalable circuit representation learning. In IEEE/ACM International Conference Computer-Aided Design (ICCAD), 2024.
Kazuyuki Takeda. OpenCores. https: oatpss///ppccooee.oog 2008.
Shailja Thakur, Baleegh Ahmad, Zhenxing Fan, Hammond Pearce, Benjamin Tan, Ramesh Karri, Brendan Dolan-Gavit, and Siddharth Garg. Benchmarking Large Language Models for Auto- mated Verilog RTL Code Generation. In IEEE/ACM Proceedings Design, Automation and Test in Eurpoe (DATE), pp. 1-6, 2023.
Yun-Da Tsai, Mingjie Liu, and Haoxing Ren. RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Models. arXiv preprint, 2024.
Vaswani. Attention is all you need. In Annual Conference on Neural Information Processing
Systems (NIPS), 2017.
Ziyi Wang, Chen Bai, Zhuolun He, Guangliang Zhang, Qiang Xu, Tsung-Yi Yi Ho. Bei Yu, and Huang. Functionality Matters Netlist Representation Learning, In ACM/IEEE Design Automation Conference (DAC), pp. 61-66, 2022.
Liangjian Wen, Yi Zhu, Lei Ye, Guojin Chen, Bei Yu, Jianzhuang Liu, and Chunjing Xu. Lay- ou Trannsffomer: Generating Layout Patterns with Transformer via Sequential Pattern Modeling. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 1-9, 2022.
Clifford Wolf, Johann Glaser, and Johannes Kepler. Yosys-a free Verilog synthesis suite, In Austrian Workshop on Microelectronics (Austrochip), 2013.
Haoyuan Wu, Zhuolun He, Xinyun Zhang, Xufeng Yao, Su Zheng, Haisheng Zheng, and Bei Yu. ChatEDA: A Large Language Model Powered Autonomous Agent for EDA. IEEE Transactions Computer-Aided Design of Integrated Circuits and Systems (TCAD), 2024.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li. Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024.
YosysHQ. PicoRV32 Size-Optimized RISC-V CPU. picorv32 2020.
Yanqing Zhang, Haoxing Ren, and Brucek Khailany, GRANNITE: Graph Neural Network Inference for Transferable Power Estimation. ACM/IEÉE Design Automation Conference (DAC), pp. 1-
6, 2020.
Haisheng Zheng, Zhuolun He, Fangzhou Liu, Zehua Pei, and Bei Yu. LSTP: A Logic Synthesis Tim- ing Predictor. In /EEE/ACM Asia and South Pacific Design Automation Conference (ASPDAC), pp. 728-778-773 223-733, 2024.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 MORE DETAILS OF MGVGA
A.1.1 LOGIC EQUIVALENCE PRESERVATION
In traditional masked graph modeling processes, nodes in graphs are masked directly and then re- constructed without any constraint except labels of masked nodes. However, any valid circuits can be labeled during the reconstruction process for circuit representation learning. It's hard for GNNs to learn useful features for downstream tasks in this way. Consequently, introduce constraints during the decoding process to ensure that GNNs learn useful features related to the circuit.
In work, "logical equivalence preservation" describes the equivalence between AIG g (V, and its different representations including X = (V, A) and Xy. In MGM, g and its latent space embedding X are logically equivalent. According to X gE(V,A), the latent space embedding X is derived from 9 through the GNN encoding process without masking gates(nodes). Consequently, we can say that X and G comes from the same truth table. In VGA, G and its corresponding Verilog code embedding Xv are logically equivalent. Xv is extracted via LLM according to the given Verilog code and G is obtained from the Verilog code via logic synthesis tools as illustrated in Figure 2 Similarly, we can say that the truth tables of Xv and G are the same.
In summary, we won't 't change the original structure of AIG G during the reconstruction process and the logic information is retained in X or Xv as the constraint for the decoding process.
A.1.2 UNMASKED ADIACENCY MATRIX
As we mentioned, following previous masked modeling paradigms (Hou et al. 2022 2023), we do not mask the adjacent matrices A during the entire process. Here are the detailed reasons.
Logical Equivalence: Our MGVGA emphasizes logical equivalence during training, ensuring that the circuit has a unique solution during reconstruction. If both A and X were masked, the problem would become NP-hard due to multiple possible solutions, making it much more difficult to train the model effectively and convergently.
Computational Complexity: Reconstructing the adjacency matrix A would require handling a matrix of size 2 for N gates, which computationally infeasible for large circuits (e.g., those with millions of gates). By not masking A, we only need to reconstruct the masked gate information in X, significantly reducing computational complexity and improving the efficiency of both training and inference. Moreover, both MGM and VGA in MGVGA operate at the gate level without masking the adjacency matrix A, focusing on local feature extraction. Consequently, we can utilize parallel processing techniques, distributed computing, etc., to reduce overhead in both MGM and VGA stages for high-complexity circuits.
In summary, perform the MGVGA without masking the adjacency matrix A, which is the same as previous masked graph modeling methods (Hou et al. 2022 2023). Consequently, there are only two tasks for gate-level prediction reconstruct masked circuits illustrated in Section 3.4
A.2 MORE DETAILS OF EXPERIMENT SETTINGS
A.2.1 AIG COLLECTION
Yosys (Wolf et al. 2013) is utilized to conduct logic synthesis, which converts source codes of circuit designs into the standardized AIG format. Moreover, we prepare 1500 optimization se- quences, each containing 20 synthesis transformations including rewrite, resub, refactor, rewrite -2, resub -2, re factor -2, and ba bance transformations, consistent with prior works (Chowdhury et al. [2022] Zheng et al., (024).. Then sequential synthesis transformations are carried out by the logic synthesis tool ABC (Brayton & Mishchenko (2100) and their correspond- ing labels are generated. Meanwhile, we also store the AIG after each synthesis transformation for AIG self-supervised learning. The resulting AIG dataset, post-technology mapping with the Nan- Gate 45nm technology library and the "5K heavy 1k" wireload model, comprises 810000 AIGs and 40500 synthesis labels across various optimization sequences and circuit designs.
14

---

Published as a conference paper at ICLR 2025
A.2.2 BASELINE SELECTION
During the baseline selection process, we acknowledged and recognized the progress made with DeepGate3 (Shi et al. 2024), the upgraded version of DeepGate2 (Shi al. 2023). However, we encountered several challenges in our practical implementation.
Specifically, DeepGate3's architecture is built upon DeepGate2 and incorporates transformer mod- els, which have a quadratic time complexity during attention computation. This becomes a critical issue when dealing with large-scale datasets. Our training set includes digital circuits with up to mil- lions of gates, leading to substantial and often unmanageable computational costs during training. During the testing phase, DeepGate3 is limited to circuits with up to thousands of gates. When at- tempting to infer circuits with millions of gates, the computational overhead becomes prohibitively high. This limitation makes it impractical for our use case, where need to handle circuits of varying sizes, including very large ones.
Consequently, we choose DeepGate2 as our baseline because it's the best model that provides more practical and scalable solution for the digital circuits we aim to model and optimize within the EDA toolchain. Moreover, we also provide some experiment results for the comparison between DeepGate3 and MGVGA on the small-scale designs with just thousands of gates in Appendix A.3
A.2.3 MORE DETAILS OF MODEL SETTINGS
Model Size. DeepGate2 (Shi al., 2023) has 0.64M parameters with 1 layer and DeepGate3 (Shi al.) (024) has 8.17M parameters with transformer architecture. Meanwhile, our MGVGA has only 0.12M parameters with 7 layers. According to the experiment results, MGVGA achieved much better performance compared to DeepGate2 and DeepGate3 with fewer model parameters, which demonstrates the effectiveness of our method.
Bidirectional LLM. We utilize gte viee-Qee2--BB BBiinstruct (Li et al. 2023b) model for Verilog code representation extraction, which is based on a BERT-like encoder transformer architecture with bidirectional attention. The base model of gte-Qween2-7B-insteee 2024). is a decoder-based model with causal attention. Qwen2-7B-instruuc has been extensively trained on a diverse corpus of Verilog and demonstrates an extraordinary ability understand and process various styles and complexities of Verilog/System Verilog code, including less standard- ized or non-optimized representations. Although it excels in generating text and understanding sequential dependencies, it is not well-suited for embedding tasks due to its unidirectional attention mechanism. Consequently, Li et al.] (2023b) proposes GTE to transform Qwen2-7B-instree into Nee--wen2--BB-inste enabling the model to capture bidirectional context while preserving the original capabilities in understanding Verilog codes.
Qwen2-7B-instruut (Yang al.,
A.2.4 BENCHMARK SELECTION
Given the practical requirements of our application, chose DeepGate2 Gate2 uses portion of the design from open-source benchmarks as training data. To ensure fair and reliable testing, excluded these designs from test set. To further validate the reliabil- ity and practicality of our method, we have supplemented our test set with designs from different opensource benchmarks (Chowdhury al. [2021] Amarú al. 2015 Open 20099] YosysHQ 2020 Asanovic 2016) that were not used during training. These additional test sets cover range of graph sizes and complexities, ensuring that our evaluation is comprehensive and representative of weal-world EDA tool requirements.
baseline. Deep-
A.2.5 EVALUATION DETAILS OF QOR PREDICTION
For the evaluation of the QoR prediction task, we evaluate the performance across ten circuit de- signs as illustrated in Table I with each circuit undergoing synthesis through 1500 optimization quences, each containing 20 steps. Notably, normalization applied to E R1x1500 representing the count of optimized gates per sequence, following (Chowdhury et al. 202T). Specifically, each element A, is standardized using A; A-A; where A and σ is the mean and the standard de- viation of A, respectively. For QoR prediction, we need to rank the predicted scores B E R x 1500 to identify the best optimization sequence. Consequently, we utilize the Normalized Discounted
15

---

Published as a conference paper at ICLR 2025
Table 6: Performance of DeepGate models and MGVGA on logic equivalence identification.
DeepGate2
DeepGate3
MGVGA (Ours)
Design
# PT
# PO
Gates
Fl Score
AUC
F1 Fl--e
AUC
F1 F1-Score
AUC
be0
21
11
2784
0.327
0.813
0,373
0.819
0.396
0.817
apexl
0.223
0.826
k2
45
45
4075
0.276
0.695
0.345
0.834
0.492
0.919
0.575
0.918
mainpla
26
49
9441
0.290
0.732
0.305
0,763
0.281
0.746
Average
0.338
0.752
0.390
0.834
0.474
0.859
Cumulative Gain (NDCG) (Järvelin & Kekäläinen| 2017 Järvelin & Kekäläinen 2002
metric to
assess the quality of the ranking algorithms for the predicted scores B. The NDCG @ k calculated as follows:
(B,,) liga(i +
Aran((A..) loga(i 1)
NDCG@K
(10)
where k represents the position considered in the ranking. Here, rank(A,)) and rank(B, denote the indices in A and B of the i-th largest elements, respectively. The NDCG@k score ranges from -1 to with a higher score indicating better ranking performance. A perfect ranking would achieve an NDCG@ @ k score of 1. Furthermore, we evaluate and compare the predictions on reference set of optimization sequences with actual synthesis labels using the Top-k% Commonality metrics, defined as (um(ArBB.) where A& and B, represent the top 1% performing optimization sequences, num At)
actual and predicted, respectively.
A.3 EXTENDED EXPERIMENTAL RESULTS
A.3.1 LOGIC EQUIVALENCE IDENTIFICATION
As shown in Table 6 we present the performance comparison between DeepGate models (Shi et al., 2023 2024) and MGVGA on small circuit designs considering the computation overhead. The experiment results show that MGVGA outperforms DeepGate3, achieving an average F1-score of 0.474 compared to 0.390, and an average AUC of 0.859 versus 0.834.
A.3.2 BOOLEAN SATISFIARILITY SOLVING
Boolean satisfiability (SAT) solving aims to determine whether there exists an assignment of truth values that satisfies a Boolean formula. In logic synthesis, SAT solvers are indispensable for tasks such as logic optimization, ensuring both the correctness and efficiency of circuits. Despite their critical role, SAT solving remains computationally challenging, often leading to significant run- time overhead, especially for large or complex designs. To address this challenge, several stud- ies (Haaswijk al. 2019 Shi et al. 2023 2024) have proposed various methods accelerate the SAT-solving process. For instance, Exact synthesis (Haaswijk et al. (019)) enhanced SAT solving by systematically varying the number of nodes and levels in directed acyclic graph (DAG) topologies, a technique referred to as Boolcan fences. A Boolean fence is a partition of nodes across multiple levels, with each level containing least one node. By tuning these parameters, they effectively constrained the search space for the SAT solver, resulting in more efficient and predictable synthesis outcomes. Despite achieving significant speedup over SAT solvers, the solution still has room for further enhancement. Building on the work presented in (Haaswijk et al. 2019). we demonstrate how MGMVA can efficiently accelerate the SAT-solving process.
Exact Synthesis generates logic circuits that guarantee logical equivalence between the resulting circuit and the target logic function, intending to find an optimal implementation based on specific criteria, such as gate count or depth.
Experiment Settings. We integrate the MGMVA into the SAT solver (Haaswijk et al., 2019) to perform exact synthesis tasks. For the training process, we apply the exact synthesis process to the EPFL (Amarú et al. 2015) and IWLS (Albrecht 2005) benchmarks using the SAT solver (Haaswijk) et al. 2019) to obtain the number of nodes and levels in the subcircuits, which subsequently used as labels. We extract the circuit embeddings from MGVGA and feed them into MLP to per- form regression tasks. As for the SAT-solving process with MGVGA, first extract several small
16

---

Published as a conference paper at ICLR 2025
Table 7: The comparison of SAT solving runtime (solver only).
Design
Subcireuits
Exact Synthesis
DeepGate2
DeepGate3
MGVGA (Ours)
Solver
Red.
Solver
Red.
Solver
Red.
adder
27
365.19
480.72
-31.64%
464.21
-27.11%
184.19
49.56%
sqrt
38
5,55
6.65
-19.82%
6.92
-24.68%
4.77
14.05%
hyp
80
328.58
351.25
-6.00%
351.71
-7.04%
213.20
35.11%
i2c
169
267.15
62.21
76.71%
34.01
87.27%
32.98
87.65%
72.83&
Average
999.99
399.41
18.24%
347.89
21.31%
255.92
53.09%
Table 8: The comparison of SAT solving runtime (overall).
Design
Exact Synthesis
DeepGate2
DeepGate: Solver
MGVGA (Ours)
Model
Solver
Overall
Model
Overall
Model
Solver
Overall
adder
365.19
0.66
480.72
464.21
475.07
0.17
184.19
184.36
sqrt
5.55
0.65
6.65
7.30
10.91
6.92
17.83
0.24
4.77
5.01
hyp 328.58
2.13
351.25
353.38
36.01
351.71
387.72
1.32
213.20
215.84
12c
267.15
2,60
62.21
64.81
45,95
34.01
79.96
1.06
32.98
34.04
div 4033.48
40,80
1096.21
1137.01
763.71
882.89
1646.60)
17.11
844.45
861.56
Average 999.99
9.37
399.41
408.78
173.49
347,89
521.38
3,99
255.92
259.91
subcircuits from the original circuits following the exact synthesis process. These subcircuits, which have a maximum of inputs and a single output, are still computationally challenging due to the exponential number of Boolean functions 2256 resulting in large runtime overhead. Then, we use the MGMVA to predict the number of nodes and levels required for the optimal equivalent im- plementation of the input subcircuit and use MGMVA to further constrain the search space of the Boolean fence, thereby accelerating the SAT-solving process. Finally, to assess the efficacy of our model in accelerating SAT solving, employ DeepGate2 (Shi et al. 2023) and DeepGate3 (Shi et al. 2024) as baselines. All experiments are conducted using the same computational resources.
Evaluation Results. Table 7 and Table & present runtime comparison among the exact synthesis, DeepGate2, DeepGate3, and our MGVGA settings, with runtime measured in seconds (s). We choose exact synthesis as our baseline setting the runtime reduction compared to the baseline setting is denoted as Red. As shown in Table 7 MGVGA achieves an average runtime reduction of 53.09% while DeepGate2 and DeepGate3 achieve an average reduction SAT solving process. However, it is worth noting that the SAT solver with GNN is less effective for easier cases, as the model inference process accounts for significant portion of the total runtime as illustrated in Table 8 In general, MGVGA exhibits significant improvement compared DeepGate models in this task, indicating that MGVGA can capture more informative abstract functional and fine-grained structural representations for solving practical SAT problems.
18.24% and 21.31 % separately for
17


Reviews:
[{"review_id": "qsutM5CXC7", "reviewer": "Reviewer_xqkr", "summary": "The paper introduce MGVGA, a constrained masked modeling framework for circuit representation learning that combines Masked Gate Modeling (MGM) and Verilog-AIG Alignment (VGA). MGM masks gates in the latent space instead of the original circuit to preserve logical equivalence. VGA uses Verilog code as a constraint, allowing graph neural networks (GNNs) to capture circuit functions alongside structural information. The framework aims to leverage the strengths of large language models (LLMs) for understanding circuit functionality from Verilog code and addresses traditional challenges in masked modeling for logic synthesis tasks. Experimental results indicate MGVGA's superior performance in Quality of Results (QoR) prediction and logic equivalence identification compared to state-of-the-art.", "questions": "1. How does MGVGA handle variations in Verilog/system verilog code style and complexity, and is there a performance threshold for handling less standardized or non-optimized Verilog representations?\n\n2. What are the computational implications of applying MGVGA to larger circuits, and are there strategies for reducing overhead in both MGM and VGA stages for high-complexity circuits?\n\n3. Could the MGVGA framework be adapted to support other types of hardware description languages (HDLs) beyond Verilog, and if so, what modifications would be necessary?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The framework introduces a novel way to apply masked modeling to circuits without compromising logical equivalence, which has been a critical limitation of prior masked modeling approaches. The combination of MGM for structure and VGA for function is effective for circuit representation.\n\n2. The paper uses Verilog-AIG alignment, that  enables GNNs to learn abstract circuit functionalities beyond structural layouts and leverages the representational power of LLMs for Verilog. This step improves the model’s ability to handle complex circuit functions.\n\n3. MGVGA demonstrates significant improvements over DeepGate2 in QoR prediction and logic equivalence identification, validating its efficacy for circuit representation tasks.", "weaknesses": "1. although the framework performs well for circuits of moderate complexity, it is unclear how well it scales to larger circuit representations, especially in terms of maintaining efficient reconstruction performance. The masking techniques could face computational bottlenecks when applied to large-scale circuit datasets.\n2. Deepgate3[1] is already release and has significantly better performance than deepgate2. The author should compare to the latest results.\n\n[1] DeepGate3: Towards Scalable Circuit Representation Learning", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UQKKnfebuU", "reviewer": "Reviewer_rsFP", "summary": "The MGVGA paper explores how to augment Masked Graph Modelling of circuits with Verilog-AIG alignment so that the resulting trained GNN-encoder will be able to encode both graph level circuit-topology information as well as higher level information from the Verilog representation.\n\nThe paper then shows its performance against DeepGate2 on various tasks -- such as logic-equivalence and QoR -- to show how MGVGA improves upon the SOTA in this field.", "questions": "\"As for the LLM, we utilize gte-Qwen2-7B-instruct (Li et al., 2023b), trained with bidirectional attention\nmechanisms based on Qwen2-7B (Yang et al., 2024), which has a comprehensive understanding of\nabstract circuit function described in Verilog codes (Liu et al., 2023b; Pei et al., 2024).\"\n\nWhat is the \"bidirectional attention mechanism\" from gte-Qwen2-7B-instruct. While I understand the utility of bidirectional attention in this context, curious to the details on the implementation -- would this perhaps be a BERT-like Encoder Transformer or perhaps a Encoder-Decoder Style approach?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This paper very clearly introduces the challenges of applying direct masked-gate-modeling for circuits as well as their decision to perform latent space masking and combining the loss function with a Verilog representation of the circuit for adding more higher-level/abstract information to the output of the encoding.\n\nThe paper evaluates against existing SOTA on logic equivalence, one of DeepGate2's tasks as well, thus a solid benchmark.", "weaknesses": "Figures 3 may benefit from additional legends and annotation perhaps indicating how the model is concurrently trained (details of the loss function).\n\nIf Figure 4 was closer to Figure 3 (perhaps below Figure 3) it would help with seeing precisely when the Verilog comes into the picture. \n\nA figure illustrating Qwen-2 and how it is augmented (or plugged-in) to the training process might be needed to clarify for the reader how this is wired in as a teacher, generally the Verilog-AIG section may benefit from additional diagrams depicting how the LLM integrates with the larger training setup.\n\nAlso the \"bidirectional attention mechanism\" may benefit from a figure or a few lines further to discuss this feature of the gte-Qwen2-7B-instruct model chosen for the Verilog-AIG.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gBrUfKJ1Xa", "reviewer": "Reviewer_jhYv", "summary": "This paper introduces MGVGA, a constrained masked modeling paradigm incorporating masked gate modeling and Verilog-AIG alignment for circuit representation learning. Specifically, MGVGA reserves logical equivalence by masking gates in the latent space rather than in the original circuits, and reconstructs masked gates under the constraints of equivalent Verilog codes. Experiments demonstrate the effectiveness of MGVGA.", "questions": "Please refer to weaknesses for details.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThis paper focuses on circuit representation learning, which is an important problem when incorporating machine learning into many downstream tasks, such as logic synthesis and physical design. \n2.\tThe idea of using the embedding of LLMs to teach GNNs seems reasonable. \n3.\tExperiments demonstrate the effectiveness of MGVGA.", "weaknesses": "1.\tThe motivation of leveraging the masked modeling paradigm for learning circuit representations is unclear. \n2.\tThe novelty of the proposed method seems incremental. The authors first apply the masked graph autoencoder to learning circuit representations, and align the learned embedding with LLMs. However, the masked graph autoencoder is a relatively mature technique, and aligning embeddings with LLMs has been widely investigated in previous work [1,2,3,4].\n3.\tExperiments are insufficient. First, some important baselines are missing. It would be more convincing if the authors could compare their method with the recent SOTA method HOGA [5], and the typical GNNs, such as GCN, GAT, etc. Second, the authors may want to conduct experiments on the OpenABC-D benchmark, which is widely used in previous work [5]. \n\n[1] Wang, Duo, et al. \"LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings.\" arXiv preprint arXiv:2408.14512 (2024).\n\n[2] Li, Yuhan, et al. \"A survey of graph meets large language model: Progress and future directions.\" arXiv preprint arXiv:2311.12399 (2023).\n\n[3] Radford, Alec, et al. \"Learning transferable visual models from natural language supervision.\" International conference on machine learning. PMLR, 2021.\n\n[4] Liu, Shengchao, et al. \"Multi-modal molecule structure–text model for text-based retrieval and editing.\" Nature Machine Intelligence 5.12 (2023): 1447-1457.\n\n[5] Deng, Chenhui, et al. \"Less is More: Hop-Wise Graph Attention for Scalable and Generalizable Learning on Circuits.\" DAC 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lGyO6CeUaY", "reviewer": "Reviewer_EYP6", "summary": "This paper consists of two methods to auto-encode an AIG circuit. The first approach (MGM) encodes an AIG with masking. Rather than masking AIG nodes, it first transform the AIG to $N$ vectors in which $N$ is the number of nodes, and then replace some of the vectors to a \"mask vector\" $m$. The second approach leverages the Verilog code of the AIG. It first masks some AIG nodes and transforms the masked AIG to another $N$ vectors, and then encode the corresponding Verilog code by LLM. The $N$ vectors and the encoded Verilog code are then merged by a neural block to obtain the AIG representation. Experimental results show that the proposed methods outperformed the baseline method (DeepGate2) on two specific tasks.", "questions": "Questions:\n1. In Figure 1, why some of the AND nodes have only a single input?\n2. In line 71, \"logical correctness can still be maintained without necessarily preserving logical equivalence\", what does \"logical correctness\" mean? It seems different from \"logical equivalence\", which might mean that the circuit is valid (like, no cycles, no NOT node has multiple inputs, etc.)\n\nSome other questions can be found in the \"Weaknesses\" section above.\n\nSuggesstions on additional contents that may help clarify how the proposed method works:\n1. Some detailed explanations/proofs/examples on how the preservation of logical equivalence is achieved.\n2. Details on the reconstruction process of the AIG (the process from $\\mathbf{X}$ to $(\\mathcal{V'}, \\mathcal{A'})$, for which the labels in Section 3.4 like $\\mathbf{D}^-$ and $\\mathbf{D}^+$ can be computed and backpropagated)\n3. How MGVGA works in inference. If the equivalent Verilog code in Figure 3 is not required, highlight it.\n4. More details in the experimental section about the proposed and compared model (especially the model size and number of layers)\n\nSome other suggestions:\n\n5. The mathematical notations may be marked on Figure 4 for clearness.\n6. Instead of identifying logic equivalence directly, it might be of more practical meaning to conduct experiments on downstream tasks requiring \"rough\" logic equivalence checking to be strictly validated later, such as SAT solving.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. As far as I know, this is the first work to integrate an LLM into circuit representation learning, and if I understand correctly, the method only requires the LLM to engage during the training stage.\n2. Improvement over the baseline method on two specific tasks (QoR prediction & logic equivalence identification)\n3. Some ablation studies are provided.", "weaknesses": "Some key information is omitted, not highlighted enough, or not presented in a clear way, which impede a thorough understanding of this paper, including\n\n1. The preservation of logical equivalence is highlighted multiple times (line 74, 90, 101, 531), but with no detailed explanation/proof/example on how this is achieved.\n2. How the AIG is reconstructed from latent space representation $\\mathbf{X}$? More specifically, if the encoder is $\\mathbf{X} = g_E(\\mathcal{V}, \\mathcal{A})$, why the decoder is not something like $(\\mathcal{V}', \\mathcal{A}') = g_D(\\mathbf{X})$, but formulated as Equation (2) instead? The adjecency matrix $\\mathcal{A}$, which I believe should be the output of a typical reconstruction process, serves as the input of Equation (2) without justification, which confuse me a lot. I also checked Section 3.4 but there are only two tasks on the reconstructed circuit, without explanation of how the reconstructed circuit comes out.\n3. In line 252, $X_V \\in \\mathbb{R}^{1\\times d_v}\\$ and it works as both key and value, does that mean there is only one key and one value in the cross attention block shown in Figure 4? It is pretty weird to have only a single key-value pair in the cross attention block, since the output will trivially be the value itself. \n3. Figure 3 shows the training process of MGVGA, however, it is not clear how MGVGA module (as shown in Figure 5) works in the inference stage. More specifically, Figure 3 includes equivalent Verilog codes for the input outputs, which I hypothesis should not be required during inference, but I cannot confirm it through the paper.\n4. In Section 4.2, while the training parameters are provided, the paper does not contain enough details of the model itself. For example, how many layers are included and what is the total number of parameters? Such details are also missing for the baseline method (DeepGate2). Generally speaking, I would like to confirm whether the size of all the compared models are close to each other. For example, in the reserach of LLM, a 7B model is usually compared with other 7B models rather than a 405B model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)