Title: SyncTREE: Fast Timing Analysis for Integrated Circuit Design through a Physics-informed Tree-based Graph Neural Network
Venue: NeurIPS 2023
Accepted: yes
Keywords: Graph Neural Networks

Integrated Circuits

Circuit Timing Analysis

Physics-guided Deep Learning
Abstract: Nowadays integrated circuits (ICs) are underpinning all major information technology innovations including the current trends of artificial intelligence (AI). Modern IC designs often involve analyses of complex phenomena (such as timing, noise, and power etc.) for tens of billions of electronic components, like resistance (R), capacitance (C), transistors and gates, interconnected in various complex structures. Those analyses often need to strike a balance between accuracy and speed as those analyses need to be carried out many times throughout the entire IC design cycles. With the advancement of AI, researchers also start to explore news ways in leveraging AI to improve those analyses. This paper focuses on one of the most important analyses, timing analysis for interconnects. Since IC interconnects can be represented as an RC-tree, a specialized graph as tree, we design a novel tree-based graph neural network, SyncTREE, to speed up the timing analysis by incorporating both the structural and physical properties of electronic circuits. Our major innovations include (1) a two-pass message-passing (bottom-up and top-down) for graph embedding, (2) a tree contrastive loss to guide learning, and (3) a closed formular-based approach to conduct fast timing.  Our experiments show that, compared to conventional GNN models, SyncTREE achieves the best timing prediction in terms of both delays and slews, all in reference to the industry golden numerical analyses results on real IC design data.

SyncTREE: Fast Timing Analysis for Integrated Circuit Design through a Physics-informed Tree-based Graph Neural Network ∗

Abstract

Nowadays integrated circuits (ICs) are underpinning all major information technology innovations including the current trends of artificial intelligence (AI). Modern IC designs often involve analyses of complex phenomena (such as timing, noise, and power etc.) for tens of billions of electronic components, like resistance (R), capacitance (C), transistors and gates, interconnected in various complex structures. Those analyses often need to strike a balance between accuracy and speed as those analyses need to be carried out many times throughout the entire IC design cycles. With the advancement of AI, researchers also start to explore news ways in leveraging AI to improve those analyses. This paper focuses on one of the most important analyses, timing analysis for interconnects. Since IC interconnects can be represented as an RC-tree, a specialized graph as tree, we design a novel tree-based graph neural network, SyncTREE, to speed up the timing analysis by incorporating both the structural and physical properties of electronic circuits. Our major innovations include (1) a two-pass message-passing (bottom-up and top-down) for graph embedding, (2) a tree contrastive loss to guide learning, and (3) a closed formular-based approach to conduct fast timing. Our experiments show that, compared to conventional GNN models, SyncTREE achieves the best timing prediction in terms of both delays and slews, all in reference to the industry golden numerical analyses results on real IC design data.

1 Introduction

Electronic design automation (EDA) tools are indispensable for designing today's complex integrated circuits (ICs) that can have multi-billion transistors (switches) and logic gates. EDA tools can be

0.7

From A

B

B

E

Supply voltage[

0.5

0.6

Pin to B

Pin to C

Interconnect

0.4

A

0.3

C

A-WW

><<

0.2

delay(A→C)

Interconnect

0.1

slewatC

0

InterconnectDesign

RCTreeCircuit

0.56

0.57

0.58

0.59

0.6

SPICEsimulation time[ns]

classified as two categories: the analysis tools that help to evaluate an IC design's quality metrics (e.g., timing performance, signal integrity, and power efficiency), and the optimization tools that use the metrics to guide the various design optimization choices such as logic synthesis, transistor sizing, gate placement, and interconnect layout optimization. Those analyses are often based on physical properties of electronic circuits, such as the Kirchhoff's Current Law (KCL), Kirchhoff Voltage Law (KVL), and electronic components constitutive equations (i.e., the voltage-current characteristics of electronic components such as resistance R, capacitance C, and transistors).

To obtain the most accurate (and golden) analysis results, the industry de factor tool is SPICE ('Simulation Program with Integrated Circuit Emphasis'), which solves a set of differential algebraic equations (DAEs) derived from KCL and KVL via numerical integration methods. Though SPICE simulation is accurate, it is also notoriously slow and not applicable to large-scale ICs' design optimization. Therefore, most EDA analyses tools resort to some mathematical approximation techniques while still follow the physical principles to solve the DAEs in order to strike a balance between accuracy and speed, because those analyses need to be carried out many times throughout the entire cycle of IC design optimization. With the advancement of AI, researchers also start to explore news ways in leveraging AI to improve those traditionally physics-based analysis methods [1].

This paper focuses on one of the most important analyses, timing analysis, for IC interconnects (a.k.a. wires) as shown in Figure 1. For a modern IC design, there are billions of on-chip interconnects that connect billions of transistors and gates, and the total wire length can be as long as hundreds of kilometers. The interconnects help to propagate electronic signals (as represented in a voltage waveform) from a driving gate (the source) to its various downstream receiving gates (the sinks), and the interconnects can be modeled as a distributed RC-tree circuits as shown in the middle of Figure 1. In RC tree, a path is composed of all nodes and edges from the driving source (e.g. A) to a specific leaf node or sink (e.g. B and C). Path delay is defined as the time delay of a voltage waveform propagating through the path measured at the 50% of the waveform's voltage level. At the leaf node, the transition time of a rising voltage waveform from the 10% voltage level to the 90% of voltage level is defined as the waveform's slew (similarly, a falling voltage waveform's slew is defined as the transition time from 90% to 10% voltage levels). Delays capture how fast the voltage waveform propagates from the source to the sinks, and slews reflect how well the propagated voltage waveform looks like at the sinks, that is why both delays and slews metrics are of utmost importance in IC design.

Since tree is a specialized graph, it seems logical to apply graph neural networks (GNNs) [2] to learn a fast timing analysis model from many of known interconnect RC-trees. However, a direct application of modern GNN models will not work well because they do not take into the physical properties of the underlying electronic circuits. For example, as mentioned in prior works [3, 4, 5, 6], most message-passing GNNs cannot retrieve global context dependencies because of the over-smoothing effect. To address this problem, in this paper, we propose a novel tree-based GNN model based on a directed two-pass message-passing mechanism, SyncTREE as shown in Figure 2 to speed up the interconnect RC-tree's timing analysis.

In summary, our major contributions are:

We formulate a novel GNN model for circuit timing prediction which can provide powerful expressiveness by incorporating global and subbranch dependencies during message passing. To better guide the graph learning process, we further design a new contrastive loss based on Circle loss [7], called Tree Contrastive (TC) loss, to take advantage of the monotone properties of timing delays along an RC path. Importantly, to our best knowledge, this is the first closed-form solution of leveraging GNNs to perform circuit timing analysis.

TopDown

Propagation

𝑙

!

...

𝑙

"

𝑙

!

𝑙

#

...

𝑨𝑮𝑮

𝒖𝒑

𝑨𝑮𝑮

𝒅𝒐𝒘𝒏

Embeddings

ℎ

'

!

Embeddings

$ ℎ

'

"

Init Nodes

RC Tree

Mask

Wire Delay/Slew

Linear

𝑹𝒆𝒂𝒅𝒐𝒖𝒕

$ ℎ

'

!

𝑙

#

𝑙

"

BottomUp

Propagation

1

ZZZ

We create two new interconnect data sets and evaluate our model on them, one synthetically generated and one extracted from the real RISC-V IC designs. According to the experiment results, SyncTREE achieves the best accuracy for both delay and slew prediction when compared with modern GNN models, such DeepGCN[8], GraphTrans[9], and NeuralTree[10]. Furthermore, the experiments validate the advantages of TC loss in guiding our model's learning over Circle loss.

We analyze the time complexity of SPICE and our model. From the time cost measurement results of the SPICE simulation and our model's inference, it is shown that our model can effectively speed up the timing process with little time increase along with circuit size. Besides, We further evaluate our model's expressiveness by transfer learning experiments on delay and slew prediction tasks. The results show that the representation learned from delay prediction task can effectively transfer to slew prediction task with better performance.

2 Related Work

GNNs in IC Design. Recently, AI for EDA has become a cutting-edge research topic. Among learning-based IC design methodologies, GNNs exhibit great potential in enhancing IC design workflows. Some recent researches leverage GNNs (1) to speed up optimization in physical design [11], layout [12], placement[13], and analog design [14]; (2) to improve the prediction analysis of net length [15], timing [16], and routing congestion [17]; and (3) to even produce configuration and guidance in floorplanning [18] and 3D integration [19].

Other Related GNNs. Despite the success of GNNs, most of them are limited to capture longrange context information. Inspired by residual neural networks (ResNets), Li et al. [8] proposed the DeepGCN model to gain a large receptive field with skip connections and dilated convolutions. This allows for the training of very deep GCNs with alleviated gradient vanishing issues. Recently, motivated by the great success of Transformers, Wu et al. [9] proposed the GraphTrans model to learn representations of long-range context. It adds a CLS token to aggregate all pairwise interactions to obtain global graph embedding, and it achieves state-of-the-art results on several graph classification tasks. Although deep GNNs can broaden the receptive fields, it may also lead to node representations that are indistinguishable across the entire graph. Some other approaches are proposed is to enhance node features by incorporating rich information of important subgraph structures. For instance, Neural Tree [10] is a special GNN model that incorporates sub-structural information into feature embeddings by performing message passing on a pre-processed H-tree instead of the original input graph. The H-tree is hierarchically organized from top to down by different subgraphs which are decomposed by the junction-tree algorithm. The benefit of H-tree is that it connects each node to

its sub-graphs which makes the Neural Tree model can learn from substructures. However, it will confuse the global structural information because of its decomposition-based methodology.

Graph Contrastive Learning. Being popular in the visual representation learning field, contrastive learning [20] aims to learn discriminative representations for positive and negative samples constructed by the data augmentation process. For graph representation, recent works [21, 22, 23, 24] try to perform contrastive learning to capture rich structure information and make features in agreement with expected transformations in a supervised manner.

3 Problem Statement

To obtain the golden results accurately for an electronic circuit, the industrial de facto simulation tool is SPICE [25], through modified nodal analysis (MNA) [26], which builds a system of algebraic differential equations (DAEs). For an arbitrary RC circuit, the DAEs can be constructed by following the physical constraints of KCL, KVL, and the electronic components' branch constitutive equations (i.e., the voltage-current characteristics of resistance R, capacitance C, and voltage sources E), and it can be written as

where C is the diagonal matrix containing all capacitance; G is the diagonal matrix of all conductance (i.e., the inverse of the resistance R ); E is the vector of voltage sources' values; A C , A R , and A V are the incidence matrices built from element types of capacitance C , resistance R , and voltage source V , respectively; i V is the unknown current vector through voltage sources, and e is the vector of unknown node voltages. The propagation delays and slews of voltage waveforms at all sink nodes can be obtained by solving the above DAEs via numerical integration methods. SPICE usually applies a time-discretization scheme (e.g., the backward Euler method) to solve the DAEs at every time step numerically, leading to particularly intensive computations and long runtime.

Problem. Path delays and slews are two critical timing metrics that guide almost every optimization step in an IC design flow, and how to compute them accurately and quickly is of utmost importance. Accurate circuit simulation through SPICE is possible, but it would take very long time. Moreover, as the IC design process is iterative with incremental changes to the interconnect's RC parameters, SPICE simulation cannot take advantage of such incremental updates. Our objective is to design a fast and accurate timing prediction model by leveraging the powerful representation ability of GNNs.

4 Physics-informed SyncTREE for Interconnect Timing Analysis

Since electronic circuits have complicated dynamics, it is a big challenge for a GNN model to represent rich structural information at different levels. The key idea of our SyncTREE model is to construct a two-pass message-passing for our tree-based graph representation.

First, we define an undirected graph G = ( V , E ) corresponding to an RC tree extracted from IC interconnects, where each vertex represents a grounded capacitance, and each edge represents the resistance between two vertices. Instead of relying on the undirected graph representation for the RCtree G = ( V , E ) , we derive two directed graph representations, G bu = ( V , E bu ) and G td = ( V , E td ) , with G bu representing the directed bottom-up tree while G td representing the directed top-down tree. For simplicity, we will use the general graph representation G = ( V , E ) to discuss the common features between G bu and G td whenever there is no ambiguity. Otherwise, we will denote them explicitly. Given a graph G = ( V , E ) , the feature of each node i ∈ V in hidden layer l is represented by vector h l i ∈ R d l . We adopt the general graph kernel from Graph Attention Networks (GAT) [27] as our message-passing backbone. We will leave the exploration of other GNN message-passing mechanisms as our future work.

4.1 SyncTREEE Overview

Figure 2 shows the overall model architecture (or algorithm) for SyncTREE. The RC-Tree circuit is represented as two separate directed graphs, G bu for the directed bottom-up tree and G td for the directed top-down tree. We first perform GAT like message-passing on the bottom-up graph G bu by

following the bottom-up directed edges. In doing so, we propagate information from the leaf nodes, through the sub-branches, toward the source nodes. The level of influences will depend on the depth of layers l L as used in GAT. At the end, every node's embedding is noted as h l L bu . We then perform GAT like message-passing on the top-down graph G td by following the top-down directed edges. Different from the bottom-up pass, we first copy the hidden feature of h l L bu from the bottom-up graph G bu to initialize the corresponding nodes' features in the top-down graph G td . After that, we apply GAT like message-passing top-down on the top-down graph G td with its own depth of layers l L . At the end, we treat the node representations of the last layer in G td as the final GNN embeddings and then feed it into the linear layer for readout.

4.2 Details of Message Passing in SyncTREE

Inside basic Message-passing layers, we use GAT as the basic block to collect information. Since we treat the node capacitance as the node's attribute and wire resistance as the edge attribute in RC tree graphs, in order to preserve wire resistance information in embeddings, we modify the aggregation mechanism inside GAT to linearly combine node feature h j and edge feature e ij with normalized attention coefficients α ij as the final output features for every node. The update of the node state after a single hidden layer follows:

where Θ and Θ e are separately applied to every node and every edge as linear transformations, || represents concatenation, T is transpose operation, and N ( i ) ⊆ V is the neighbor node set of i .

In SyncTREE, we respectively applied GAT bu and GAT td to perform message passing on the bottom-up tree G bu and the top-down tree G td , respectively.

The feature representation in G bu is updated by h l +1 i,bu = AGG l bu ( { ( h l j,bu , e ij,bu ) | j ∈ N ( i ) ∪ i } ) . When we start the top-down message-passing, we use the aggregation output of the final layer in G bu to initialize the node features in G td :

For every convolutional layer in G td , the node hidden states will be updated by two parts: node representations updated from the previous layer of G td itself and the copied final embedding from G bu . It can be expressed as:

Since AGG bu and AGG td are performed on two directed graphs, SyncTREE can effectively avoid over-smoothing and preserve variance among nodes. In our case, we aim at predicting path timing, thus we use the final predictions at leaf nodes (sinks) by applying a mask to calculate the loss function.

4.3 Tree Contrastive Loss

It is a well-known fact that the voltage along an RC path is a monotonic function of distance [28]. For a target node, in its RC path to the root node, its timing would be similar to its nearest node and have bigger differences with further nodes. Inspired by contrastive learning, we introduce tree contrastive (TContrast) loss to enhance node embedding quality by optimizing the pair similarity of node representations. The idea behind contrastive learning is to pull positive samples close to the target node and push the negative samples away.

At every batch during training, we randomly sample a set of target leaf nodes with size B . For any target leaf node i ∈ B , we get a node embedding set S i by picking the final representations h L i,td of positive and negative nodes from V . To determine positive and negative samples of a target leaf node, we set a hops hyperparameter. On the RC path from the target node to the source node, we treat the nodes within the hops to the target as positive samples and the others as negative samples.

It should contain M positive samples and N negative samples, which means that there will be M pair of within-class similarity s p and N pairs of between-class similarities s n . We apply Euclidean distance to measure similarity as follows:

where α is used to avoid s ij being too small. For pair similarity optimization, it is natural to maximize within-class similarity s p and minimize between-class similarity s n . A novel optimization manner of Circle loss is to reduce ( α n s n -α p s p ) , where the α n and α p are independent weight factors. The intuition behind it is different similarity scores should have different penalty strengths. For the Circle loss, it aims to optimize s p → 1 and s n → 0 , and use m the radius of the circle decision boundary [7]. However, it's unfair for the Circle loss to use the same relaxation strength m to s p and s n . In TContrast loss, we design a more flexible decision boundary based on the priori path resistance of samples. For each ( s k p , s q n ) pair, we define a coefficient r k,q d of path resistances between positive node k to target i and negative node q to target i as follows:

where the numerator/denominator is the sum of resistance in RC path i to k / q . Instead of only using similarity score to amplify the gradient to s p and s n , we separately set distance-adaptive scaling factors based on path resistance for α n and α p by:

Following the setting in original Circle loss, we can deduce the TContrast loss of node i as:

Since the decision boundary is achieved at α n ( s n -m ) -α p ( s p -(1 -m )) = 0 , combined with (7), the decision boundary in TContrast loss is given by:

It shows that the decision boundary of TContrast loss is an ellipse arc, where the center is (0 , 1) , and the lengths of the semi-axis along s n and s p are m · √ 1 + e 2 r d -1 and m · √ 1 + e 1 -2 r d . So TContrast loss expects s n < m · √ 1 + e 2 r d -1 and s p > 1 -m · √ 1 + e 1 -2 r d . The intuitive interpretation is that we set different relaxations to s n and s p according to r d . If r d increases, it will have a more strict margin for s p and a looser margin for s n , which means we emphasize on improving s p when the negative nodes close to target.

In TContrast loss L TC , We formulate different penalty strengths according to the similarity score and relative distances of node pairs. We serve it as a regularizer in our final objective loss function, which also includes L1-loss L l 1 of true timing and predicted timing:

5 Experimental Setting

In this section, we first explain the datasets, parameter settings, and baselines of our experiments, then demonstrate the advantages of our method in the IC timing prediction through the experiments.

Prediction Task. Given the RC trees extracted from IC interconnections and the driving voltage at the input node, we aim to predict the pin-to-pin timing including delays and slews. We perform the prediction tasks on two benchmark datasets, which are summarized in Table 1, to comprehensively evaluate the performance of our SyncTREE model. Given SPICE timing results as golden, we use

MAE as the evaluation metric for both delay and slew predictions. All experiments in this paper are implemented with PyTorch 1.13.1 and PyTorch Geometric 2.2.0 frameworks and executed on a Ubuntu server equipped with Intel Xeon Gold 6330 CPU with 56 cores/2 threads running at 2.0GHz. The reference SPICE simulations are carried out with the commercial Synopsys HSPICE simulator on an AMD Ryzen 3950X with 16 cores/32 threads at 3.5GHz. (Code and datasets are available at https://github.com/xlab-ub/SyncTree).

Synthesized Dataset. The dataset consists of synthesized RC circuits with various circuit typologies, including rare interconnections. To generate these circuits, we first generate an RC tree and then simulate its corresponding artificial IC interconnects with SPICE to obtain the golden timing values. The pseudocode for generating RC trees, the workflow for data preparation, and the golden timing distribution in data samples are provided in the Appendix.

RISC-V dataset. The dataset is composed of real RC circuits extracted from practical RISC-V IC designs. Our objective is to validate the effectiveness of our model in analyzing the timing of interconnects in practical IC designs.

Models and parameters. For our SyncTREE model, we separately set 32 hidden dimensions on the synthetic dataset and 128 hidden dimensions on the RISC-V dataset. The TContrast loss is set with hops of 2, N of 1, m in the range of [0 . 1 , 0 . 5] , and B is 64 for the synthetic dataset and 128 for the RISC-V dataset. All the models are trained with the Adam optimizer ( β 1 = 0 . 9 , β 2 = 0 . 99) . The batch size is set to 32 for the synthetic dataset and 256 for RISC-V dataset. The learning rate was set to 8e-4 for 4/8/16-layer models and 4e-4/2e-4 for 32/64-layer models. All the models are trained for 10 epochs on the RISC-V dataset and 60 epochs on the synthetic dataset.

Baselines. Considering timing analysis is a unique task that depends on the structures of RC trees at different levels, we compare our method to various conventional GNNs, including GCN [29], GAT [27], and GraphSAGE [30], as well as prospective GNNs such as DeepGCN[8], GraphTrans[9], and NeuralTree[10]. For GraphTrans, to incorporate complex global information into node features, we concatenate the CLS token embedding of Transformer with node embeddings and input to MLP to get the final output.

6 Results and Discussion

In this section, we present a comprehensive evaluation of our SyncTree model's performance compared to selected GNN baselines for delay and slew prediction tasks and showcase the impact of using TContrast loss function for training. Then, we discuss the computational efficiency of SyncTREE and its transfer learning capabilities.

Main Results. We present the delay and slew prediction results of different GNN models on the Synthetic and RISC-V datasets in Table 2 and Table 3. The results show that SyncTREE outperforms the existing GNN baselines across all model depths on both synthetic and RISC-V datasets. For example, even the only four-layer SyncTREE model nearly surpasses all the other baselines with different model depths. Furthermore, our model consistently outperforms the baselines on the RISC-V dataset, highlighting its applicability to real-world scenarios. To evaluate the effectiveness of graph representations obtained by different GNN models, we visualize the final node embeddings of the converged GNNs in Figure 5 and calculate the Pearson correlation coefficient with the target delay distribution across the entire circuit graph. The results show that our model can achieve the highest correlation score compared to other baselines, further validating its superiority.

(a) MAE on S

(b) Percent error on S

(c) MAE on R

(d) Percent error on R

40

20

Delay Mean Average Error(ps)

40

品

0000420

100

Delay Relative Error(%)

0.15

Delay Mean Average Error(ps)

2

00004020

100

Delay Relative Error(%)

Circuit Size

30

15

Circuit Size

Circuit Size

0.10

Circuit Size

10

10

10

20

0.05

369121518

Number of Leaf Node

3691215

Number of Leaf Node

18

Number of Leaf Node

3

6

6

Number of Leaf Node

3

6

6

(a) MAE on S

(b) Percent error on S

(c) MAE on R

(d) Percent error on R

Circuit Size

100

Circuit Size

40

品

0000420

100

Slew Relative Error(%)

Circuit Size

2

15

0.6

Mean Average Error(ps)

Circuit Size

20

15

15

Relative Error(%)

00

55025

00

0.4

10

10

10

20

0.2

5

3691215

Number of Leaf Node

18

3691215

Number of Leaf Node

18

Number of Leaf Node

2

4

6

8

10

Number of Leaf Node

2

4

6

8.10

Accuracy vs. circuit size and the number of sinks. To provide more insights into the performance of our method, we investigate the relationship between prediction accuracy and two important factors: circuit size and the number of sinks (i.e., leaf nodes) in RC trees. As depicted in Figure 3 and Figure 4, our method achieves better accuracy on circuits with larger sizes and more sinks.

TContrast loss vs. Circle loss. We provide the prediction results of SyncTREE models which trained with TContrast (TC) loss and Circle (C) loss in Table 2 and Table 3, we found that SyncTREE+TC performs better than the default L1 loss and Circle loss, highlighting the advantages of TC in optimizing similarity pairs and loss optimization convergence. To further compare the original

(a) Target

(b) Ours(

0.94

)

(c) GAT(

0.82

)

(d) NTREE(

0.67

)

0.41

)

(a) Circle Loss

(b) TContrast loss

1.0

1.0

0.8

0.8

0.6

S

0.6

0.4

0.4

0.2

0.2

0.0

0.5

1.0

0.0

0.5

1.0

Sn

Sn

Circle loss and TContrast loss, we analyzed their convergence status in the delay prediction task. Figure 6 shows the distribution of similarity pairs ( s n , s p ) after convergence. We observed that the similarity pairs using TContrast loss are closer to the optimization goal and have a more concentrated distribution in similarity space.

Computational Efficiency. SPICE typically requires significant computational resources to model circuit behavior in a time-incremental manner. However, considering that RC circuits are linear systems, we can utilize linear multi-step numerical integration techniques like the Trapezoidal method [31] to solve the differential-algebraic equations if the form of dx dt = Ax inside SPICE for transient simulation. This results would be x ( t + h ) = ( I -1 2 hA ) -1 ( I + 1 2 hA ) x ( t ) , given a circuit with n nodes and a driving signal with m time steps from the source. With LU factorization being used to invert n × n matrices, the computation complexity of SPICE is expressed as O ( m · ( n 3 + n 2 + n )) .

In contrast, the computational complexity of our SyncTREE model is significantly reduced, which makes it highly efficient. As it's GAT-based, there are no extensive matrix operations required, and the computations can be parallelized easily. Moreover, the number of layers l is less than n . Thus, the time complexity of our model can be expressed as O ( l · ( n · f · f ′ +( n -1) · f ′ )) , where f is the number of input features, f ′ is the number of output features, and n -1 is the number of edges. We provide the computational efficiency comparison of SyncTREE with SPICE simulation in Figure 7. Our results indicate that SyncTREE is significantly faster than SPICE in obtaining accurate results, with the advantage increasing as the circuit size grows.

Transfer Learning. We provide the results of ablation study of transfer learning of SyncTREE on both delay and slew tasks in Table 4. During transfer learning, we froze the weights pre-trained on the other task and only fine-tuned the readout module on the target task. The results show that the representations learned from the delay prediction task can be efficiently transferred to the slew task, resulting in better performance. The indicates that SyncTREE's learned representation can be efficiently transferred between tasks, which can save time and resources by avoiding the need for additional training on every new target task from scratch.

7 Conclusion

In this paper, we propose a novel GNN model to predict circuit timing with a much faster computation speed than the SPICE simulator. Furthermore, based on the RC tree's structural and physical properties, we devise a tree contrastive (TContrast) loss to guide the feature transformation during graph learning. The results show that our model reaches state-of-the-art performance compared

with prospective GNNs and outperforms the SPICE simulator in computation efficiency. Therefore, our SyncTREE model can better support fast incremental timing updates. Compared to SPICE's full-fledged simulation even for incremental circuit updates, our solution's incremental updates will be even more appealing to IC design optimization.

Acknowledgment

This work was supported, in part, by the SUNY-IBM AI Collaborative Research Alliance Project #22003, the SUNY Empire Innovation Program, and, by Advantest as part of the Graduate School "Intelligent Methods for Test and Reliability" (GS-IMTR) at the University of Stuttgart.

References

Guyue Huang, Jingbo Hu, Yifan He, Jialong Liu, Mingyuan Ma, Zhaoyang Shen, Juejian Wu, Yuanfan Xu, Hengrui Zhang, Kai Zhong, et al. Machine learning for electronic design automation: A survey. ACM Transactions on Design Automation of Electronic Systems (TODAES) , 26(5):1-46, 2021.

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems , 32(1):4-24, 2020.

Haoxing Ren, Siddhartha Nath, Yanqing Zhang, Hao Chen, and Mingjie Liu. Why are graph neural networks effective for eda problems? In Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design , pages 1-8, 2022.

Zizheng Guo, Mingjie Liu, Jiaqi Gu, Shuhan Zhang, David Z Pan, and Yibo Lin. A timing engine inspired graph neural network model for pre-routing slack prediction. In Proceedings of the 59th ACM/IEEE Design Automation Conference , pages 1207-1212, 2022.

Mingjie Liu, Walker J Turner, George F Kokai, Brucek Khailany, David Z Pan, and Haoxing Ren. Parasitic-aware analog circuit sizing with graph neural networks and bayesian optimization. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE) , pages 1372-1377. IEEE, 2021.

Daniela Sánchez Lopera, Lorenzo Servadei, Gamze Naz Kiprit, Souvik Hazra, Robert Wille, and Wolfgang Ecker. A survey of graph neural networks for electronic design automation. In 2021 ACM/IEEE 3rd Workshop on Machine Learning for CAD (MLCAD) , pages 1-6. IEEE, 2021.

Yifan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, and Yichen Wei. Circle loss: A unified perspective of pair similarity optimization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 6398-6407, 2020.

Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. Deepgcns: Can gcns go as deep as cnns? In Proceedings of the IEEE/CVF international conference on computer vision , pages 9267-9276, 2019.

Zhanghao Wu, Paras Jain, Matthew Wright, Azalia Mirhoseini, Joseph E Gonzalez, and Ion Stoica. Representing long-range context for graph neural networks with global attention. In Advances in Neural Information Processing Systems (NeurIPS) , 2021.

Rajat Talak, Siyi Hu, Lisa Peng, and Luca Carlone. Neural trees for learning on graphs. Advances in Neural Information Processing Systems , 34:26395-26408, 2021.

Yi-Chen Lu and Sung Kyu Lim. On advancing physical design using graph neural networks. In Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design , pages 1-7, 2022.

Haoxing Ren, George F Kokai, Walker J Turner, and Ting-Sheng Ku. Paragraph: Layout parasitics and device parameter prediction using graph neural networks. In 2020 57th ACM/IEEE Design Automation Conference (DAC) , pages 1-6. IEEE, 2020.

Yaguang Li, Yishuang Lin, Meghna Madhusudan, Arvind Sharma, Wenbin Xu, Sachin S Sapatnekar, Ramesh Harjani, and Jiang Hu. A customized graph neural network model for guiding analog ic placement. In Proceedings of the 39th International Conference on ComputerAided Design , pages 1-9, 2020.

Kourosh Hakhamaneshi, Marcel Nassar, Mariano Phielipp, Pieter Abbeel, and Vladimir Stojanovic. Pretraining graph neural networks for few-shot analog circuit modeling and design. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2022.

Zhiyao Xie, Rongjian Liang, Xiaoqing Xu, Jiang Hu, Yixiao Duan, and Yiran Chen. Net2: A graph attention network method customized for pre-placement net length estimation. In Proceedings of the 26th Asia and South Pacific Design Automation Conference , pages 671-677, 2021.

Kevin Kai-Chun Chang, Chun-Yao Chiang, Pei-Yu Lee, and Iris Hui-Ru Jiang. Timing macro modeling with graph neural networks. In Proceedings of the 59th ACM/IEEE Design Automation Conference , pages 1219-1224, 2022.

Robert Kirby, Saad Godil, Rajarshi Roy, and Bryan Catanzaro. Congestionnet: Routing congestion prediction using deep graph neural networks. In 2019 IFIP/IEEE 27th International Conference on Very Large Scale Integration (VLSI-SoC) , pages 217-222. IEEE, 2019.

Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Wenjie Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Azade Nazi, et al. A graph placement methodology for fast chip design. Nature , 594(7862):207-212, 2021.

Yi-Chen Lu, Sai Surya Kiran Pentapati, Lingjun Zhu, Kambiz Samadi, and Sung Kyu Lim. Tp-gnn: A graph neural network framework for tier partitioning in monolithic 3d ics. In 2020 57th ACM/IEEE Design Automation Conference (DAC) , pages 1-6. IEEE, 2020.

Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in neural information processing systems , 33:5812-5823, 2020.

Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. Adversarial graph augmentation to improve graph contrastive learning. Advances in Neural Information Processing Systems , 34:15920-15933, 2021.

Zekun Tong, Yuxuan Liang, Henghui Ding, Yongxing Dai, Xinke Li, and Changhu Wang. Directed graph contrastive learning. Advances in Neural Information Processing Systems , 34:19580-19593, 2021.

Yuyang Wang, Jianren Wang, Zhonglin Cao, and Amir Barati Farimani. Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence , 4(3):279287, 2022.

Yiyue Qian, Chunhui Zhang, Yiming Zhang, Qianlong Wen, Yanfang Ye, and Chuxu Zhang. Co-modality graph contrastive learning for imbalanced node classification. Advances in Neural Information Processing Systems , 35:15862-15874, 2022.

Laurence W. Nagel and D.O. Pederson. Spice (simulation program with integrated circuit emphasis). Technical Report UCB/ERL M382, EECS Department, University of California, Berkeley, Apr 1973.

Chung-Wen Ho, Albert Ruehli, and Pierce Brennan. The modified nodal approach to network analysis. IEEE Transactions on circuits and systems , 22(6):504-509, 1975.

Petar Veliˇ ckovi´ c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903 , 2017.

Jorge Rubinstein, Paul Penfield, and Mark A Horowitz. Signal delay in rc tree networks. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2(3):202-211, 1983.

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 , 2016.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems , 30, 2017.

Lloyd N Trefethen and JAC Weideman. The exponentially convergent trapezoidal rule. siam REVIEW , 56(3):385-458, 2014.

A Appendix

A.1 Message Passing in SyncTREE

As the message passing process in Figure 8, information from leaves, sub-branches, and the whole global structure is first collected following the bottom-up propagation by GAT bu . Then, the final node representations of GAT bu are applied to each layer of GAT td to jointly update the node attributes of the corresponding top-down tree with the node embeddings of its previous layer. As the example shown in Figure 8, by designing this two-pass message-passing mechanism, the node features will incorporate the information from different levels and become more expressive. Furthermore, in the top-down tree, the root node can only be updated with synchronized h L bu since it doesn't have any incoming connection, it ensures that information injection at the source of the top-down tree is fixed which can help to maintain differentiable feature embeddings without over-smoothing.

1-hop

2-hop

init node

sync

ℎ

!"

#

Global

Sub-branch

Sub-branch

Leaf

Leaf

bottom-up

top-down

top-down

ℎ

!"

#

ℎ

!"

#

ℎ

!"

#

ℎ

$%

&

ℎ

$%

'

ℎ

$%

(

A.2 Synthetic and RISC-V Dataset Preparation

Our dataset is composed of artificially generated and practical RC trees and the golden timing results at sinks (leaf nodes of each RC tree) obtained by SPICE simulation. On the one hand, we follow the pipeline in Figure 9 to generate the synthetic dataset. To be specific, we first adopt Algorithm 1 to generate RC-trees with random typologies and then convert them to artificial IC interconnects for further SPICE timing measurement. On the other hand, we directly extract RC trees from practical RISC-V circuit designs to compose the RISC-V dataset.

A.3 Baselines's Implementation

The GNNs of all the baseline models are set with 32/128 hidden dimensions separately for the synthetic dataset/RISC-V dataset. For GraphTrans, the dimension of the feedforward full-connection layers in the Transformer of GraphTrans is set to 256 with 0.1 dropout probability between layers, the number of attention heads is set to 4, and the max input sequence length is set to the maximum circuit size. It should be noted that we only made a little modification to the GraphTrans model. GraphTrans

is originally designed for node classification tasks, it takes CLS token from Transformer output as the representation of the whole graph and applies a linear module followed by softmax to implement prediction. In order to incorporate global information into node features, in our experiments, we concatenate the CLS token with node embeddings and then feed it into MLP to get the final output. For NTREE, we set GAT as its basic block with a 0.2 dropout probability between layers. We follow the original junction-tree-based algorithm in [10] to compose H-trees from our RC circuits with the same radius setting for extracting subgraphs in the paper.

A.4 Analysis of TContrast Loss To visualize the converging process during training, we plot the distribution of similarity pairs in space at different epochs in Figure 10. It obviously shows that our model approaches the optimization goal with a more concentrated similarity distribution after enough training with the guidance of TContrast loss. In Figure 11, we show the MAE difference of timing results obtained by vanilla SyncTREE and TC-loss guided SyncTREE. As shown in the results, after being combined with TC loss, our SyncTREE model has smaller errors for most types of RC trees which can effectively prove the validity of TC loss.

(a) Epoch=1

(b) Epoch=20

(c) Epoch=40

(d) Epoch=60

1.0

1.0

1.0

1.00

0.8

0.8

0.8

0.75

S

0.6

0.6

S

0.6

0.50

0.4

0.4

0.4

0.2

0.25

0.2

0.2

0.0

0.00

0.00

0.25

0.50

0.0

0.5

1.0

0.0

0.5

1.0

0.0

0.5

1.0

Sn

Sn

Sn

Sn

(a) Delay on S

(b) Slew on S

(c) Delay on R

(d) Slew on R

15

30

0.015

0.100

0

10

Delay MAE Difference (ps)

0

2010。

0.010

sd)

0.075

Slew MAE Difference (ps)

Difference

0.050

Circuit Size

30

Circuit Size

另

Circuit Size

0.005

Circuit Size

0.025

&

10

0.000

0.025

0.000

-10

050°0-

1

0.075°

15

30

0.015

0.100

3

Number of Leaf Node

6

6

12

15

18

D

3

Number of Leaf Node

6

6

12

15

18

C

Number of Leaf Node

6

3

Number of Leaf Node

6

9

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis papers proposes SyncTREE that uses a bottom-up and top-down graph attention network with a Tree Contrastive Loss to predict the delay and slew for IC interconnects. Compared to other GNN methods, the proposed one achieve lower prediction error across synthetic and RISC-V benchmarks.\n\nSTRENGTHS:\n1.\tThe proposed method is quite novel and leverages the prior knowledge in timing analysis in network design and loss function design, achieving the best results on both synthetic and RISV benchmarks.\n\nWEAKNESSES:\n1.\tThe modified aggregation mechanism claims to preserve resistance information by linearly combining node and edge features. However, no ablation study or evidence on this technique has been shown. Similarly, the network designs, e.g., residual connections from bottom-up tree, two directed graphs, or other introduced techniques are not evaluated through ablation studies. It seems that the benefits are mainly from the directed graphs, not GAT. How does this method perform if applied to other GNNs listed in Table 3?\n2.\tThe layers in the trained GAT is fixed, not adaptive to different circuits. If a RC tree is deeper than the GAT network, then the timing information cannot be propagated from source to sink. The generalization raises some concerns in this case.\n3.\tIn line 266, it claims the proposed method achieves better accuracy on larger circuits. However, Figure 4(d) seems the relative prediction error get worse with larger circuit sizes if I understand correctly. Why the performance gets worse as circuit size changes is not explained. How to solve this generalization gap should be a key question to investigate for this work.\n4.\tThe runtime comparison need to clarify the hardware platform. If the GAT uses GPU, then it might be a unfair comparison if compared to CPU-version SPICE. There exists GPU version LU factorization to accelerate matrix inversion.\n5.\tThe prediction error lacks intuitive analysis. Whether 0.05 ps MAE considered to be large or small enough for timing analysis is not clear. In other words, the significance of the results is unclear. How does this delay and slew error ultimately affect critical path delay or total negative slack should be discussed. The variance in the predicted error on each node is also important compared to MAE.\n6.\tIn terms of generalization and transfer learning, transferring to different benchmark/circuit are expected instead of transfer from delay to slew prediction. How does the physics-informed loss function helps to increase data efficiency compared to pure data-drive method is not shown. The generalization and data efficiency are critical concerns for ML-based PDE solving tasks, which are not deeply explored in this paper.\n\nQUESTIONS:\nMajor questions are summarized above. Basically, more ablation studies are required. More discussion on generalization to different circuits and data efficiency is expected. I will consider increasing the score if the major concerns above are addressed.\n\nLIMITATIONS:\nNo.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-27"}

comments: SUMMARY:
This manuscript present a SyncTree to speed up timing analysis in IC design.

STRENGTHS:
-The problem that the manuscript is trying to address is important (increase speed of timing analysis)
-Evaluation and comparison to other machine learning based methods.

WEAKNESSES:
- It is unclear are the Mean Average Error with respect to Spice simulation?
- Table 2 and Table 3 should consist of time it takes to perform the predictions. 
- Additionally, what is the increase in speed between SyncTree and spice simulation.

QUESTIONS:
Check Weakness section

LIMITATIONS:
Check Weakness section
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-05

comments: SUMMARY:
This paper proposes a GNN based method that specializes in timing analysis.

STRENGTHS:
* This paper is well written and organized, easy for readers to follow.
* Related background, related work that uses GNN on circuits are discussed with sufficient level of detail.
* The core problem looks well formatted.
* Experimental results looks promising with benchmarks on RISC-V, offering faster than SPICE simulation looks appealing.
* Performance and results are thoroughly analyzed.

WEAKNESSES:
Please see questions

QUESTIONS:
* For different circuit technology sizes, does the proposed method need to be trained differently? or is the technology sizing part of the input?

LIMITATIONS:
* There is no explicit discussion on limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a GNN model, dubbed SyncTREE, for IC's RC-tree timing analysis. Two techniques are proposed: 1) two-pass message-passing and 2) Tree Contrastive loss. Experiments of two IC designs demonstrate the best accuracy of SyncTREE over other SOTA GNN models.\n\nSTRENGTHS:\n1. Domain-specific knowledge is used in SyncTREE's two techniques. I think these two techniques are original.\n\n2. Evaluations demonstrate the performance of SyncTREE's two techniques. The timing analysis shows the promising potential speedup of SyncTREE over traditional SPICE with increased circuit sizes.\n\nWEAKNESSES:\n1. There is no clear guideline on setting the number of hidden dimensions and layer selection, which can influence accuracy and speed trade-offs for the practical adoption of the proposed SyncGNN. \n\n2. I'm pondering if an EDA-focused conference like DAC or ICCAD may be a more suitable platform for this paper. Given that the development of SyncGNN draws heavily on EDA-related domain-specific knowledge, such a conference could provide a more targeted audience and potentially foster more fruitful discussions for further refining this approach.\n\nQUESTIONS:\n1. I am confused by the middle part of Fig. 2 w/ \"Init Nodes\". The hidden features from the bottom-up graph are used to initialize those in the top-down graph. Why there is an arrow from the top-down graph to the \"Init Nodes\"? In addition, the right-most part of Fig. 2 shows a \"mask\", which is given w/o explanation.  \n\n2. What is the rule of thumb for setting the number of hidden dimensions? In addition, it would be better if the authors can discuss the selection of the number of layers. These hyperparameters can influence the accuracy and speed trade-offs and a discussion can potentially ease the use of the proposed SyncTREE. \n\n3. Fig.7 provides the computation time vs. circuit size. What are the hardware platforms for SPICE simulation and SyncTREE model? Since given a new circuit, new graph training is needed. It would be better if the author can also provide the training time vs. circuit size for better show the efficiency of the proposed SyncTREE. \n\n4. Typos. E.g., \"news ways\" should be \"new ways\" in the Abstract section. Line # 216, \"We\" should be \"we\". \n\n5. Considering that timing analysis is an important problem for EDA community and considering existing efforts to use GNN for EDA tasks, can conferences such as DAC and ICCAD be a more suitable platform for publishing this paper?\n\nLIMITATIONS:\nThe authors do not discuss the limitations of the proposed method. It would be better if the authors can provide the limitations and the potential future directions of this work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-30"}