Title: Circuit as Set of Points
Venue: NeurIPS 2023
Accepted: yes
Keywords: EDA

Circuit Design

Congestion prediction

DRC violation prediction
Abstract: As the size of circuit designs continues to grow rapidly, artificial intelligence technologies are being extensively used in Electronic Design Automation (EDA) to assist with circuit design.
Placement and routing are the most time-consuming parts of the physical design process, and how to quickly evaluate the placement has become a hot research topic. 
Prior works either transformed circuit designs into images using hand-crafted methods and then used Convolutional Neural Networks (CNN) to extract features, which are limited by the quality of the hand-crafted methods and could not achieve end-to-end training, or treated the circuit design as a graph structure and used Graph Neural Networks (GNN) to extract features, which require time-consuming preprocessing.
In our work, we propose a novel perspective for circuit design by treating circuit components as point clouds and using Transformer-based point cloud perception methods to extract features from the circuit. This approach enables direct feature extraction from raw data without any preprocessing, allows for end-to-end training, and results in high performance.
Experimental results show that our method achieves state-of-the-art performance in congestion prediction tasks on both the CircuitNet and ISPD2015 datasets, as well as in design rule check (DRC) violation prediction tasks on the CircuitNet dataset.
Our method establishes a bridge between the relatively mature point cloud perception methods and the fast-developing EDA algorithms, enabling us to leverage more collective intelligence to solve this task. To facilitate the research of open EDA design, source codes and pre-trained models are released at https://github.com/hustvl/circuitformer.

Circuit as Set of Points

Abstract

As the size of circuit designs continues to grow rapidly, artificial intelligence technologies are being extensively used in Electronic Design Automation (EDA) to assist with circuit design. Placement and routing are the most time-consuming parts of the physical design process, and how to quickly evaluate the placement has become a hot research topic. Prior works either transformed circuit designs into images using hand-crafted methods and then used Convolutional Neural Networks (CNN) to extract features, which are limited by the quality of the handcrafted methods and could not achieve end-to-end training, or treated the circuit design as a graph structure and used Graph Neural Networks (GNN) to extract features, which require time-consuming preprocessing. In our work, we propose a novel perspective for circuit design by treating circuit components as point clouds and using Transformer-based point cloud perception methods to extract features from the circuit. This approach enables direct feature extraction from raw data without any preprocessing, allows for end-to-end training, and results in high performance. Experimental results show that our method achieves state-of-the-art performance in congestion prediction tasks on both the CircuitNet and ISPD2015 datasets, as well as in design rule check (DRC) violation prediction tasks on the CircuitNet dataset. Our method establishes a bridge between the relatively mature point cloud perception methods and the fast-developing EDA algorithms, enabling us to leverage more collective intelligence to solve this task. To facilitate the research of open EDA design, source codes and pre-trained models are released at https://github.com/hustvl/circuitformer .

1 Introduction

In recent years, the demand for semiconductor chips for various applications has been increasing, and the need to accelerate the chip design and manufacturing process has become increasingly urgent. And with the development of semiconductor technology, the scale of very-large-scale integrated (VLSI) circuits is growing exponentially, challenging the scalability and reliability of the IC design process. As a result, the industry urgently needs more efficient Electronic Design Automation (EDA) algorithms and tools to enable the processing of huge exploration spaces in a shorter period of time. The layout and routing phase are particularly important in the overall EDA flow. As feature size shrinks, routability becomes a more important constraint on the manufacturability of VLSI designs. Routing congestion can significantly affect PPA (Performance, Power, Area) metrics, and design rule violations can directly lead to unmanufactured designs, but there is no way to obtain an accurate congestion and design rule violations map without conducting placement and routing attempts, which can lead to longer design cycles and possibly some unpleasant surprises. Therefore, accurate and rapid evaluation of placement in advance of routing will be very important, and the exploration of routability becomes one of the key issues in modern placement for VLSI circuits [27].

Before global placement, the circuit design is represented as a netlist consisting of nodes and nets, where nodes represent specific components and nets represent link relationships between nodes, containing topological information about the design. After the global placement, we will get information about the specific location of each node, containing the geometric information. How to use this information to make a fast and accurate evaluation of the placement is currently divided into roughly two approaches: (1) using hand-crafted features to rasterize the circuit and thus convert them into images. Then, using CNN [24, 38] or Generative Adversarial Nets (GAN) [42, 1, 45] to perform feature extraction on the circuit. (2) converting the circuit design into a graph structure and using Graph Neural Networks (GNN) [20, 35, 15] to perform feature extraction on the circuit design.

However, both methods have their advantages and disadvantages. The CNN-based methods use some hand-crafted features (e.g., RUDY [33], node density, pin density) to rasterize the circuit and perform feature extraction on it, where the grids are treated as pixels. Then, use classical image segmentation networks in computer vision (e.g., UNet [32] and FCN [25]) to do the pixel-wise prediction of labels (e.g., congestion and DRC violation). This method is simple and efficient, however, the effectiveness of the hand-crafted features limits the network's performance, and it can not achieve end-to-end training. The GNN-based methods involves converting the circuit design into a graph with vertices and edges, where nodes in the circuit are represented as vertices and nets are represented as edges. GNN networks are then used to extract features from the graph, making it suitable for downstream tasks. Although they have small parameters and fast inference speeds, it is worth noting that even though the time consumption of constructing the graph structure has been proven to be linearly related to the design scale, the design scale of chips is very large. The number of nodes is typically in the hundreds of thousands, while the number of edges can be in the millions, so the process of constructing the graph structure can often be very time-consuming. In addition, these methods also use hand-crafted features when processing geometric information, which also suffers from the aforementioned problems.

To address the above issues, in this paper, we approach the task of feature extraction in circuit design from a novel perspective by treating circuit components as point clouds and using a Transformer to solve the task, i.e., CircuitFormer. CircuitFormer avoids the two core pain points of the above two methods: (1) there is no need to use hand-crafted features, and end-to-end training can be achieved; (2) there is no need for any preprocessing, which greatly reduces the time of the entire process.

As shown in Fig. 1, our model is divided into two parts: an encoder and a decoder . The encoder extracts features from the placement information of the circuit design, while the decoder adapts to downstream tasks using a conventional backbone and segmentation head, similar to the CNN-based methods.

n × 𝑑

𝑤×ℎ×𝑙

n × 𝑑

𝑤

2 ×

ℎ

2

×𝑙

ResNet-18

UNet++

input

GA

GA

𝑤

4 ×

ℎ

4

×𝑙

n × 𝑑

𝑤

8

×

ℎ

8

×𝑙

GA

GA

n × 𝑑

CA

CA

CA

CA

CA

CA

CA

CA

In encoder , in order to model the relationship between nodes, to compensate to some extent the disadvantage that geometric methods cannot use topological information, and to enable the exchange of information across the grids, we introduce the attention mechanism. Considering that most perception tasks for point clouds are focused on 3D point clouds, but circuits can be abstracted as 2D point clouds without a Z-axis, we propose an improvement to the voxel-based set attention (VSA)

module [16], adapting it for use in 2D scenarios to extract 2D node features. In Sec. 3.1 we will provide a detailed introduction.

The decoder consists of a backbone and a segmentation head. We use ResNet-18 [17] as the backbone to further refine the rasterized features extracted by the encoder. Finally, we attach a UNet++ [46] as the segmentation head to make pixel-wise predictions.

Our experiments were based on the publicly available CircuitNet [5] dataset and ISPD2015 [4] dataset. Whether it is the congestion prediction task or the DRC violation prediction task, the pixel values of their labels are continuous values between 0 and 1. The label distribution is highly imbalanced, exhibiting a skewed distribution with a long tail. The imbalance of label distribution can make it difficult for a model to handle corner cases, leading to overfitting and reducing the accuracy and robustness of the model. For the regression on imbalanced datasets, we adopted label distribution smoothing (LDS) [39] to re-weight the loss function and improve the model's performance.

In summary, our contributions are:

Our work takes a novel perspective on circuit design by treating circuit components as point clouds and using a point cloud perception network to extract features for circuit design. To the best of our knowledge, our work is the first to abstract the task of circuit design feature extraction as a point cloud perception task. This bridges the gap between the two fields, allowing us to transfer certain structures and conclusions from mature point cloud perception methods to the developing EDA domain.

Our method surpasses hand-crafted features in terms of feature extraction and enables end-to-end training, making it superior to CNN-based methods. Additionally, unlike GNNbased methods, our approach eliminates the need for preprocessing, significantly reducing processing time.

Experiments validated the superior performance of our method on the CircuitNet and ISPD2015 datasets. In the congestion prediction task, our method surpassed the baseline [24] by an average of 3 . 8% and 1 . 5% on the CircuitNet dataset and ISPD2015 dataset respectively, based on the average of Pearson, Spearman, and Kendall correlation coefficients, achieving a new SOTA. For the DRC violation prediction task, we also achieved the best performance in most of the metrics. Our work has the potential to serve as a universal feature extractor for chip design, which can adapt to a wider range of downstream tasks by changing the decoder.

2 Related Work

2.1 CNN-based Methods

The core idea of CNN-based methods is to partition the circuit design into grids and extract features of the components in each grid using some hand-crafted methods. These methods treat the circuit as an image and the grids as pixels. Then, [24, 38] use features as the input to the segmentation network and performs pixel-wise prediction. [42, 1, 45] transform the task into generating a congestion map from features and uses a Conditional Generative Adversarial Network (CGAN) [28] to solve it.

2.2 GNN-based Methods

The GNN-based methods primarily focus on the topology of circuit design, which involves reconstructing the circuit design as a graph with vertices and edges. In [21], nodes serve as vertices, while nets serve as edges, then GAT [35] is used to extract features. [9] uses GraphSAGE [15] to extract features from a graph with G-cells as nodes. These methods can only utilize the topological information of the circuit design and are unable to handle geometric information. Recent advancements in graph networks have expanded their capabilities to incorporate geometric information in addition to topological information. LHNN [36] considers the grid as internal nodes and the net as external nodes, constructing a homogeneous graph. CircuitGNN [40] generates a heterogeneous graph, treating both nodes and nets as vertices, with information being propagated along both topological and geometric edges.

2.3 Point Cloud Perception

Point cloud perception methods can be classified into two categories: point-based and voxel-based. Point-based methods [30, 31] follow the paradigm of sample, group, and fusion. They perform farthest point sampling in the point cloud, then group points within a certain range of each sampled point, and finally perform the fusion. These methods can preserve the structure of the point cloud without losing information. However, due to the irregularity of point clouds, such methods are often difficult to train in parallel and require a large amount of memory consumption. Voxel-based methods [44] assign each point to a specific voxel and extract local features from groups of points within each voxel. This significantly improves computational efficiency but can result in information loss. Transformers can model the relationships between points and have vast potential in point cloud perception tasks. However, due to the quadratic computational complexity of transformers, vanilla transformers cannot be directly applied to point clouds. Based on this, [43] performs local attention on a set of points after sampling to extract local point cloud relationships. [16] applies cross attention twice between the latent code and point cloud to compress the features into the latent space, reducing computational complexity.

3 Method

We focus on the congestion prediction task and design rule check (DRC) violation prediction task. Congestion is defined as the overflow of routing demand over available routing resources in the routing stage of the back-end design, and DRC violation, as the name suggests, refers to the location where design rules are violated. Both tasks can be abstracted as making grid-level predictions of the circuit based on the point-wise information (geometric and topological) of the circuit design. Specifically, we consider circuit components such as buffers, inverters, registers, and intellectual property (IP) cores as point clouds. Here, we do not differentiate between component categories but only focus on their fundamental geometric features, i.e., the center coordinates, width, and height of each circuit component.

3.1 Encoder

Q

n × 𝑘 × 𝑑

K

V

Q

Cross Attention

latent codes

𝑤 × ℎ × 𝑙

ConvTensor

ConvFFN

Broadcast

n × 𝑘 × 𝑑

𝑤 × ℎ × 𝑙

K

V

Input

n × 𝑑

n × 𝑑

Output

Cross Attention

Multi-scale Grid-based Attention. Due to the complexity of circuit designs, two nodes connected by a topological structure may be far apart in geometric positions. The hand-crafted method that only considers the geometric features of each node within a grid cannot handle the relationships between different grids. Therefore, we hope to introduce the attention mechanism to expand the receptive field, model long-range dependencies, and enable interaction between different grids. However, vanilla selfattention computation is quadratic, with a computational complexity of O ( n 2 d ) , where n represents the number of nodes, which is usually in the hundreds of thousands. Such high computational complexity is unacceptable. Inspired by [19, 18, 23, 16], we use a fixed-length, learnable latent code to perform two cross-attentions with the input features instead of one self-attention. This compresses the node information into a more compact latent space and significantly reduces the computational complexity. The validity of this method comes from the observation that the attention matrix is usually low-rank [11, 2].

The grid-based attention (GA) module is an improvement over the voxel-based set attention (VSA) module [16], adapted for use with the circuit, which is referred to as pseudo-2D point clouds in our work. The structure is shown in Fig. 2. Given an input set P ∈ R n × 4 where n represents the number of nodes. The features of each node can be represented as { P i = ( x i , y i , w i , h i ) : i = 1 , ...n }, x i and y i represent the original coordinates, while w i and h i represent the width and height of the component, respectively. And we also have information on the position of corresponding points C ∈ R n × 2 , C = { C i = ( ˜ x i , ˜ y i ) : i = 1 , ...n }, where ˜ x i and ˜ y i represent the normalized coordinates of the nodes. We encode the node features as keys K ∈ R n × d and values V ∈ R n × d through linear projection, same as vanilla Transformer. Randomly initialized learnable latent codes serve as queries Q ∈ R k × d ( k ≪ n ) and are fused with linearly projected node features to encode them into the latent space using cross-attention. After that, we obtain the point-wise feature H ∈ R n × k × d .

Then, we convert the point-wise features to grid-wise features, the grid coordinates corresponding to each point can be represented as G = { G i = ([ ˜ x i d x ] , [ ˜ y i d y ] : i = 1 , ...n ) }, where d x , d y is the grid size of two dimensions, and [ · ] is the floor function. By changing d x , d y , we can obtain raster features of different scales, so that we can perform multi-scale grid information exchange. To be more precise, we established four distinct grid scales, for d x = d y = [1 , 2 , 4 , 8] . Next, we use the scatter-sum function to add point-wise features belonging to the same grid and obtain the grid-wise feature A ∈ R m × k × d , where m represents the number of non-empty grids. The scatter function is a CUDA kernel library that performs symmetric reduction on various matrix segments. The entire process is highly parallelized.

To introduce local inductive bias, based on the coordinates of the grids, we construct a sparse tensor I ∈ R w × h × l by assembling the grid-wise features into a image-like structure, where w and h respectively represent the number of grids in the two dimensions, and l = k × d . To be specific, we pad the empty grids with zeros. After that, the feature I is refined using a convolutional feed-forward network (ConvFFN) while enabling information exchange between grids, we use two layers of depth-wise convolution (DWConv) to achieve it. Datasets in circuit design are typically small in scale, and this approach of combining attention mechanisms with CNNs has been shown to be effective on small datasets [29]. This is because ConvFFN can introduce inductive bias to the model, which plays an important role in improving the convergence of transformer on small datasets, which are difficult to optimize.

To maintain consistency with the input dimensions, we broadcast the sparse tensor I to generate the point-wise feature ˆ H ∈ R n × k × d based on the grid indices. Finally, the refined point-wise feature ˆ H is used as the key-value pair, with the linear projection of the input serving as the query for cross-attention. This results in an output O ∈ R n × d with the same dimensions as the input.

The encoder consists of four grid-based attention modules with different grid scales, which can aggregate multi-scale and global geometric information in a highly parallelized manner. It has a computational complexity of O ( nkd ) , where k ≪ n . At the end of the encoder, we use the scatter-sum operator to generate grid-wise features and combine them into an image Y ∈ R W × H × D according to the position coordinates of each grid, preparing it for further pixel-wise prediction. Here, Y has the same width and height as the label.

3.2 Decoder

We did not put much emphasis on the design of the decoder. We utilized a ResNet-18 [17] as the backbone to further refine and downsample the features extracted by the encoder, and then employed a classic segmentation head, UNet++ [46], to achieve pixel-wise prediction.

3.3 Loss Function with Label Distribution Smoothing

Our label smoothing strategy follows the idea of re-weighting in imbalanced classification problems [3]. However, in the case where the labels are continuous values, the empirical label distribution may not reflect the true label distribution. This is because, unlike classification tasks, there are no hard boundaries between adjacent label values. When presented with limited samples for certain labels, the model prefers to learn features of labels with similar values.

Inspired by [39], we count the empirical density distribution of the labels in the training set, and then, convolve with it using a symmetric Gaussian kernel function to obtain the smoothed effective label

density distribution. The inverse of the square root of the smoothed distribution is used as the weight to do the re-weight of the loss function. The label distribution smooth calculation is shown below:

where p ( y ) is the empirical density function of the label, we discretize the labels with a precision of 0.001 and count the frequency of the value y to obtain p ( y ) . ˜ p ( y ′ ) is the effective density function of the label y ′ , and the weight corresponding to a label with value y ′ is 1 √ ˜ p ( y ′ ) . k ( y, y ′ ) is symmetric

Gaussian kernel function: k ( y, y ′ ) ∝ e ( y -y ′ ) 2 2 σ 2 . We use MSE loss, so the re-weighted loss function is given by Eq. (2), where ˆ y is the predicted value of the network and y ′ is the value of the label.

4 Experiments

4.1 Tasks and Datasets

We evaluate our model on public CircuitNet [5] and ISPD2015 [4] datasets. For the CircuitNet dataset, we focus on the two cross-stage prediction tasks, congestion prediction and design rule check (DRC) violation prediction . For the ISPD2015 dataset, our main focus lies in its congestion prediction task.

4.2 Baselines and Settings

We compare our work with two mainstream methods in this field: CNN-based methods and GNNbased methods. The CNN-based method mostly consists of hand-crafted features as input, extracting features using the backbone, and finally adding a segmentation header for pixel-wise prediction. CircuitNet dataset provides Gpdl [24] and RouteNet [38] as baselines for congestion prediction and DRC violation prediction tasks, respectively. Gpdl is a typical CNN-based EDA-customized deep learning model for congestion prediction, it combines RUDY map, pin RUDY map, and macro region as input features. Similarly, RouteNet is a network customized for DRC violation prediction. In addition to the three hand-crafted features mentioned above, it also incorporates cell density and congestion map as input. In this paper, input features of all CNN-based methods follow these settings. For the GNN-based method, we chose the most outstanding models in the field: CircuitGNN [40], a heterogeneous graph method where topological and geometrical information are integrated jointly. We evaluate the results in Pearson/Spearman/Kendall correlation as with [40, 14].

The network is trained end-to-end on a single NVIDIA RTX 3090 GPU, for 100 epochs with a cosine annealing decay learning rate schedule [26] and 10-epoch warmup. We use the AdamW optimizer with learning rate γ = 0 . 001 .

4.3 Result of Congestion Prediction

CircuitNet. The results are summarized in Table 1. The result shows that our method achieves the best performance, and outperforms the baseline model Gpdl provided by CircuitNet, with an average metric improvement of 3 . 8% . Our method beats the UNet++ ( 2 . 8% on average), which means that the features extracted by our method are superior to the hand-crafted features, as our decoder is the same as UNet++. The performance of CircuitGNN on CircuitNet is not very good, possibly because CircuitGNN requires node-level labels for training, but the CircuitNet dataset only provides grid-level labels. We assigned the grid label to each node within the grid as the node label. During validation, we performed average pooling on the outputs of nodes within the same grid to obtain the grid-level output, which is consistent with the setting in CircuitGNN.

Fig. 3 visualizes the predicted congestion map generated by different methods. It can be intuitively seen that our method can provide more detailed predictions for the congestion map.

ISPD2015. The results are summarized in Table 2, it indicates that our model still achieves the best performance on the ISPD2015 dataset. It is worth noting that the ISPD2015 dataset used in our study

is provided by the developers of the CircuitNet dataset. However, the format they provide is not easily compatible with CircuitGNN, especially when dealing with large-scale designs. Consequently, a significant amount of time is required for data conversion in such cases. Therefore, when comparing with CircuitGNN, we excluded the superblue design from ISPD2015, which contains over one million nodes. The experimental results are presented in Table 3, similarly, we also showcase the performance of CNN-based models.

4.4 Result of DRC violation prediction

Table 4 shows that in the DRC violation prediction task, our work achieved the best performance in terms of Spearman and Kendall, but the Pearson correlation coefficient is relatively low. We speculate that this is because hand-crafted methods use the congestion map as an input feature, which contains higher-level information than the raw data. In contrast, our method only uses the original positional information as input to maintain consistency.

4.5 Ablation study

We conducted a series of ablation experiments to demonstrate the roles of different components in our model.

Comparison with other point cloud perception methods. Table 5 shows a comparison between our method and other point cloud perception methods, where we replace our encoder with them. We compared point-based methods, such as Point-Transformer [43] and Msg-Transformer [13], and voxel-based (or grid-based for 2D circuit) methods, such as PointPillars [22] and DSVT [37]. The results show that the voxel-based method outperforms the point-based method in both tasks, which

(a) Input

(b) UNet++

(c) CircuitFormer (ours)

(d) Ground-truth

may be due to the fact that the voxel-based method directly partitions the circuit design into grids, which is consistent with the labeling modality, whereas the point-based method needs to project the point-wise information to the grid-wise prediction, which may result in information loss. Additionally, the point-based method requires point sampling and cannot utilize information from all points. Our method also outperforms PointPillars, which groups points within the same grid and extracts features using MLP and Maxpooling like PointNet [30]. Similar to RUDY, this method can only extract local features within the grid and cannot exchange features across grids. It highlights the importance of information exchange across grids.

Label distribution smooth. Table 6 investigated the improvement brought by the label distribution smooth module on imbalanced prediction tasks.

Preprocessing time. In Fig. 4 we show the relationship between the preprocessing time of various methods and the size of the circuit design. GNN-based methods, represented by CircuitGNN, require generating a computation graph based on the geometric and topological information of the circuit design during the preprocessing stage. The time of this process is proportional to the size of the circuit design. CNN-based methods require hand-crafted methods to extract geometric information, which involves traversing the position of each node and pin. This process is proportional to the number of

nodes and pins. In contrast, our method requires no preprocessing and takes the raw position of each node as input directly. The time tests were conducted on AMD EPYC 7502P 32-Core 2.5GHz CPU.

50000

100000

150000

200000

250000

number of edges

0

20

40

60

80

time (s)

CircuitGNN

Hand-Crafted

CircuitFormer (ours)

Balancing model performance and runtime memory. To delve deeper into the relationship between the performance of our model and its runtime memory, and to provide more options to the community, we conducted ablation experiments on the encoder of our model. As shown in Fig. 1, our encoder consists of four stages of GA modules. We tested the performance and runtime memory of models with different numbers of stages. Specifically, we test runtime memory on RISCY-FPU-a, which is the largest-scale design in the CircuitNet dataset and consists of 77,707 nodes. The results are summarized in Table 7.

Decoder. We conducted ablation experiments on the decoder selection to explore its impact on the model's performance.

5 Conclusion and Future Work

We present a new perspective on circuit design, treating circuit components as point clouds. Based on this, we propose an improved multi-scale grid-based attention module based on VoxelSet [16], which achieves SOTA results on CircuitNet and ISPD2015 datasets. Furthermore, our work can serve as a general-purpose feature extractor for chip design and can be adapted to a wider range of downstream tasks. This is the first work to transform circuit design evaluation into a point cloud perception task, presenting a novel approach to addressing the issue. It has established a bridge between the relatively mature point cloud perception methods and the still developing EDA methods, enabling us to apply methods and conclusions that have been proven effective in point cloud perception tasks to EDA tasks.

However, it should be noted that although our approach can model the relationship between points through the attention mechanism, it still does not use the prior knowledge of topology directly. Graphormer [41] encodes the topological relationships between nodes into the attention matrix, which enables the fusion of topological and geometric information in the transformer. How to incorporate the prior topological information into our model will be our future research direction.

Last but not least, the methods we use are simple and basic ones, while the GNN-based or CNN-based methods we compare against have been iterated in EDA for several years. However, thanks to the active Transformer community, there are many more advanced methods available. For example, LongNet [10] and RetNet [34] not only reduce the computational complexity to linear but also improve accuracy. With the rise of Large Language Model (LLM), the use of Transformers for handling long sequential data (corresponding to large-scale circuit designs in the EDA field) has become a trend. There has been a wealth of research dedicated to improving the performance of Transformers while reducing computational complexity. We hope to bring this trend into the EDA domain through our work, leveraging the advancements in Transformers to enhance circuit design tasks.

Acknowledgement : This project was supported by National Natural Science Foundation of China (NSFC No. 62276108).

References

Mohamed Baker Alawieh, Wuxi Li, Yibo Lin, Love Singhal, Mahesh A Iyer, and David Z Pan. Highdefinition routing congestion prediction for large-scale fpgas. In 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC) , pages 26-31. IEEE, 2020.

Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank Reddi, and Sanjiv Kumar. Low-rank bottleneck in multi-head attention models. In International Conference on Machine Learning , pages 864-873. PMLR, 2020.

Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks. Neural networks , 106:249-259, 2018.

Ismail S Bustany, David Chinnery, Joseph R Shinnerl, and Vladimir Yutsis. Ispd 2015 benchmarks with fence regions and routing blockages for detailed-routing-driven placement. In Proceedings of the 2015 Symposium on International Symposium on Physical Design , pages 157-164, 2015.

Zhuomin Chai, Yuxiang Zhao, Wei Liu, Yibo Lin, Runsheng Wang, and Ru Huang. Circuitnet: An open-source dataset for machine learning in vlsi cad applications with improved domain-specific evaluation

metric and learning strategies. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2023.

Abhishek Chaurasia and Eugenio Culurciello. Linknet: Exploiting encoder representations for efficient semantic segmentation. In 2017 IEEE visual communications and image processing (VCIP) , pages 1-4. IEEE, 2017.

Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. arxiv 2017. arXiv preprint arXiv:1706.05587 , 2, 2019.

Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV) , pages 801-818, 2018.

Xuan Chen, Zhixiong Di, Wei Wu, Qiang Wu, Jiangyi Shi, and Quanyuan Feng. Detailed routing short violation prediction using graph-based deep learning model. IEEE Transactions on Circuits and Systems II: Express Briefs , 69(2):564-568, 2021.

Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, and Furu Wei. Longnet: Scaling transformers to 1,000,000,000 tokens. arXiv preprint arXiv:2307.02486 , 2023.

Yihe Dong, Jean-Baptiste Cordonnier, and Andreas Loukas. Attention is not all you need: Pure attention loses rank doubly exponentially with depth. In International Conference on Machine Learning , pages 2793-2803. PMLR, 2021.

Tongle Fan, Guanglei Wang, Yan Li, and Hongrui Wang. Ma-net: A multi-scale attention network for liver and tumor segmentation. IEEE Access , 8:179656-179665, 2020.

Jiemin Fang, Lingxi Xie, Xinggang Wang, Xiaopeng Zhang, Wenyu Liu, and Qi Tian. Msg-transformer: Exchanging local spatial information by manipulating messenger tokens. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12063-12072, 2022.

Amur Ghose, Vincent Zhang, Yingxue Zhang, Dong Li, Wulong Liu, and Mark Coates. Generalizable cross-graph embedding for gnn-based congestion prediction. In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD) , pages 1-9. IEEE, 2021.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems , 30, 2017.

Chenhang He, Ruihuang Li, Shuai Li, and Lei Zhang. Voxel set transformer: A set-to-set approach to 3d object detection from point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8417-8427, 2022.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, et al. Perceiver io: A general architecture for structured inputs & outputs. arXiv preprint arXiv:2107.14795 , 2021.

Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention. In International conference on machine learning , pages 46514664. PMLR, 2021.

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 , 2016.

Robert Kirby, Saad Godil, Rajarshi Roy, and Bryan Catanzaro. Congestionnet: Routing congestion prediction using deep graph neural networks. In 2019 IFIP/IEEE 27th International Conference on Very Large Scale Integration (VLSI-SoC) , pages 217-222. IEEE, 2019.

Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12697-12705, 2019.

Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. In International conference on machine learning , pages 3744-3753. PMLR, 2019.

Siting Liu, Qi Sun, Peiyu Liao, Yibo Lin, and Bei Yu. Global placement with deep learning-enabled explicit routability optimization. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE) , pages 1821-1824. IEEE, 2021.

Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3431-3440, 2015.

Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 , 2016.

Igor L Markov, Jin Hu, and Myung-Chul Kim. Progress and challenges in vlsi placement research. In Proceedings of the International Conference on Computer-Aided Design , pages 275-282, 2012.

Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 , 2014.

Namuk Park and Songkuk Kim. How do vision transformers work? arXiv preprint arXiv:2202.06709 , 2022.

Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 652-660, 2017.

Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems , 30, 2017.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 , pages 234-241. Springer, 2015.

Peter Spindler and Frank M Johannes. Fast and accurate routing demand estimation for efficient routabilitydriven placement. In 2007 Design, Automation & Test in Europe Conference & Exhibition , pages 1-6. IEEE, 2007.

Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. Retentive network: A successor to transformer for large language models. arXiv preprint arXiv:2307.08621 , 2023.

Petar Veliˇ ckovi´ c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903 , 2017.

Bowen Wang, Guibao Shen, Dong Li, Jianye Hao, Wulong Liu, Yu Huang, Hongzhong Wu, Yibo Lin, Guangyong Chen, and Pheng Ann Heng. Lhnn: Lattice hypergraph neural network for vlsi congestion prediction. In Proceedings of the 59th ACM/IEEE Design Automation Conference , pages 1297-1302, 2022.

Haiyang Wang, Chen Shi, Shaoshuai Shi, Meng Lei, Sen Wang, Di He, Bernt Schiele, and Liwei Wang. Dsvt: Dynamic sparse voxel transformer with rotated sets. arXiv preprint arXiv:2301.06051 , 2023.

Zhiyao Xie, Yu-Hung Huang, Guan-Qi Fang, Haoxing Ren, Shao-Yun Fang, Yiran Chen, and Jiang Hu. Routenet: Routability prediction for mixed-size designs using convolutional neural network. In 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) , pages 1-8. IEEE, 2018.

Yuzhe Yang, Kaiwen Zha, Yingcong Chen, Hao Wang, and Dina Katabi. Delving into deep imbalanced regression. In International Conference on Machine Learning , pages 11842-11851. PMLR, 2021.

Zhihao Yang, Dong Li, Yingxueff Zhang, Zhanguang Zhang, Guojie Song, Jianye Hao, et al. Versatile multi-stage graph neural network for circuit representation. Advances in Neural Information Processing Systems , 35:20313-20324, 2022.

Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems , 34:28877-28888, 2021.

Cunxi Yu and Zhiru Zhang. Painting on placement: Forecasting routing congestion using conditional generative adversarial nets. In Proceedings of the 56th Annual Design Automation Conference 2019 , pages 1-6, 2019.

Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision , pages 16259-16268, 2021.

Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4490-4499, 2018.

Zhonghua Zhou, Ziran Zhu, Jianli Chen, Yuzhe Ma, Bei Yu, Tsung-Yi Ho, Guy Lemieux, and Andre Ivanov. Congestion-aware global routing using deep convolutional generative adversarial networks. In 2019 ACM/IEEE 1st Workshop on Machine Learning for CAD (MLCAD) , pages 1-6. IEEE, 2019.

Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang. Unet++: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4 , pages 3-11. Springer, 2018.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel perspective on circuit design by treating circuit components as point clouds and utilizing point cloud perception methods for feature extraction. Previous approaches either relied on hand-crafted methods to transform circuit designs into images for CNN feature extraction or employed GNNs for graph-based feature extraction, which required time-consuming preprocessing. In contrast, the proposed method enables direct feature extraction from raw data without preprocessing, allowing for end-to-end training and achieving high performance. Experimental results on the CircuitNet dataset demonstrate the superiority of the proposed method in congestion prediction and design rule check (DRC) violation prediction tasks. This approach establishes a connection between point cloud perception methods and EDA algorithms, facilitating the application of collective intelligence in solving circuit design tasks. The method outperforms hand-crafted features and CNN-based methods in feature extraction while eliminating the need for preprocessing, thereby reducing processing time. The work shows potential as a universal feature extractor for chip design, adaptable to a wide range of downstream tasks.\n\nSTRENGTHS:\nThe paper introduces a groundbreaking method that addresses circuit congestion prediction and design rule check (DRC) violation prediction in electronic design automation (EDA). By leveraging the concept of point clouds and incorporating a genetic algorithm (GA) module, the proposed approach achieves significant improvements over existing techniques commonly employed in EDA tools. The utilization of point cloud perception enables a fresh perspective on circuit design analysis, allowing for more accurate and efficient prediction of congestion and violations. Additionally, the integration of the GA module enhances the optimization process, leading to enhanced results and performance. This novel method showcases the potential for innovative approaches in EDA, opening doors to advanced techniques that can better handle circuit design challenges and contribute to improved circuit performance and reliability.\n\nWEAKNESSES:\nTo enhance the comprehensiveness of the paper, it would be valuable to expand the evaluation to encompass other applications within the domain of electronic design automation (EDA), such as circuit placement. By investigating the effectiveness of the proposed method in various EDA tasks, including placement, the authors can provide a more holistic understanding of its capabilities and potential impact.\nFurthermore, conducting additional ablation studies that delve into the runtime and scaling model trade-offs would contribute to a deeper analysis of the proposed approach. These studies would shed light on the performance and efficiency aspects of the method, allowing for a better understanding of its behavior under different conditions and dataset sizes. This information would be particularly useful for practitioners and researchers aiming to apply the method in real-world scenarios.\nAnother aspect worth exploring is the authors' emphasis on the encoder while leaving the decoder section with limited analysis. This raises intriguing questions regarding the possibility of modifying the output format to potentially improve the EDA software pipeline. Investigating alternative output formats or exploring ways to optimize the decoder component could uncover opportunities for further enhancing the proposed method's integration into EDA workflows.\nMoreover, the paper opens up avenues for considering the incorporation of this approach into other areas of EDA tools beyond congestion prediction and DRC violation prediction. Exploring how the proposed method could be leveraged in other stages of the EDA process, such as routing, power optimization, or physical design verification, would demonstrate its versatility and potential for broader application. This could foster innovation in multiple facets of EDA, leading to more efficient and advanced chip designs.\nBy addressing these suggestions, the paper would offer a more comprehensive evaluation, deeper insights into runtime and scaling trade-offs, considerations for optimizing the decoder section, and opportunities for applying the proposed approach in various EDA tool domains. These extensions would further solidify the paper's contribution and provide valuable guidance for future research and development in the EDA field.\n\nQUESTIONS:\nA few questions:\n\nIn figure 5 shows, 3x model size than unet. Which creates the question of scaling model. If a smaller version of the model with same size of Unet will get better accuracy?\n\nIn figure 4 shows preprocessing times, what is the total runtime and inference time of the model?\n\nLIMITATIONS:\nThe authors addressed the limitations correctly.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-18"}

comments: SUMMARY:
This paper proposes to use point cloud perception methods to extract features from the circuit by treating the circuit components as point clouds. The proposed method solves the infeasibility of end-to-end training in the CNN-based method and the limitation of time-consuming preprocessing in the GNN-based method. The evaluation result shows the effectiveness of the proposed method on CircuitNet dataset: it outperforms the mainstream CNN-based and GNN-based methods on congestion prediction and design rule check violation prediction.

STRENGTHS:
1. This paper facilitates Electronic Design Automation (EDA) by using the well-researched point cloud perception methods, providing a new perspective for circuit design.
2. The encoder part, especially the multi-scale grid-based attention architecture, reduces the computational complexity, and compresses the information as well.

WEAKNESSES:
1. Please give the runtime latency comparison with the CNN-based method to prove the proposed method reduces design cycles.

QUESTIONS:
1. In the decoder part, motivations for using ResNet18 and UNet++ are missing. Is the selection of network architecture in the decoder part based on the downstream tasks? For different tasks, is necessary to use different networks?

LIMITATIONS:
The authors addressed limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new perspective on circuit design by viewing circuit components as point clouds and using point cloud perception methods to extract features from circuits. This achieves the goals of directly extracting features from raw data, end-to-end training, and high performance. Experimental results show that the method achieves state-of-the-art performance on congestion prediction and design rule checking tasks.\n\nSTRENGTHS:\na. Overall summary:\nThe authors propose a novel approach for chip design and manufacturing processes using Electronic Design Automation (EDA) algorithms and tools.\n\nb. Methods:\nThe authors treat circuit components as point clouds and use point cloud perception methods for feature extraction.\nThey utilize a grid-based attention (GA) module to aggregate multi-scale and global geometric information.\nThe encoder converts point-wise features to grid-wise features and generates grid-wise features using a scatter-sum operator.\nThe decoder refines and downsamples the features extracted by the encoder using a ResNet-18 backbone and a UNet++ segmentation head.\nThe loss function incorporates label distribution smoothing to address imbalanced classification problems.\n\nc.  Experimental results:\nThe authors conducted experiments on the CircuitNet dataset.\nThe proposed method achieved state-of-the-art performance in congestion prediction and design rule check (DRC) violation prediction tasks.\nThe method outperformed UNet in terms of performance, with a runtime memory of 3721MB for the proposed method and 1931MB for UNet.\n\nWEAKNESSES:\nN/A\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nThe limitations is not explicitly discussed, but the future work has plotted the current limitation and proposed the next direction that authors will try to tackle such limitation.\nNo negative societal impact", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript presents several claims. Firstly, it introduces a novel perspective on \"point cloud perception methods\" for performing EDA tasks. By treating circuit components as point clouds and utilizing point cloud perception networks, the paper proposes an innovative approach to solving these tasks. Secondly, it claims superiority over CNN and GNN-based methods commonly used for EDA tasks. The authors assert that their method outperforms these existing techniques. Thirdly, the paper positions itself as the state-of-the-art solution for EDA tasks. The experimental section is well-written, easy to understand, and the overall presentation flow of the manuscript is good.\n\nSTRENGTHS:\nIn terms of originality, paper uses novel perspective on the solving the EDA problems which novelty in the paper. clear description of the method, and well-structured experimental section, which includes an ablation study to enhance the validity\n\nWEAKNESSES:\nThe experimental results provided in the paper are not sufficient. For instance, Table1 shows the comparison of congestion with CNN and GNN methods. However, in case of CNN the cited paper uses ISPD2015 dataset (Open source in circuitnet), while this paper uses circuitnet. It may be better first show comparison with Circuitnet paper \nhttps://www.sciengine.com/SCIS/doi/10.1007/s11432-022-3571-8\nSimilarly, CircuitGNN[39] which uses the ISPD2011 dataset while this paper uses Circuitnet dataset. Maybe add another comparison point cloud perception methods vs CircuitGNN on ISPD 2011 dataset.\n\nQUESTIONS:\n1. Add more comparison SOTA GNN & CNN based method comparisons for both congestion and DRC violation prediction\n\n2. Show results on other open-source datasets like (ISPD2015 which is available in circuitnet with grid level labels) and DAC 2012.\n\n3. The paper mentions “performance is higher without excessive GPU memory”  but the memory required here is approximately double that Unet. If authors could show the comparison of memories size of graph and memory required by your method and SOTA methods would be fair.\n\n4. The paper mentions that “novel perspective on circuit design” and “work has the potential to serve as a universal feature extractor for chip design” But only showed two tasks. To make these claim needs to perform other tasks in circuit design like power prediction, and delay prediction.\n\nLIMITATIONS:\n•\tThe scope of idea is may be limited by the size of the graph. For example, if the graph contains millions of nodes and edges which requires large GPU memory to process.\n•\tGeneralization of the method presented in paper is explored it might be the limitation since it only works on grid-based dataset. But GNN is more generalized to solve EDA problems since netlist is a graph.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}