SlowGuess/NeurIPS_2023_OCR huggingface dataset:
SyncTREE: Fast Timing Analysis for Integrated Circuit Design through a Physics-informed Tree-based Graph Neural Network*
Yuting Hu
Buffalo, NY, USA
yhu54@buffalo.edu
Florian Klemme
klemme@iti.uni-stuttgart.de
Jiajie Li
jli433@buffalo.edu
Gi-Joon Nam
gnam@us.ibm.com
Tengfei Ma
Stony Brook, NY, USA
tengfei.ma@stonybrook.edu
Hussam Amrouch
München, Germany
amrouch@tum.de
Jinjun Xiong
jinjun@buffalo.edu
Nowadays integrated circuits (ICs) are underpinning all major information technology innovations including the current trends of artificial intelligence (AI). Modern IC designs often involve analyses of complex phenomena (such as timing, noise, and power etc.) for tens of billions of electronic components, like resistance (R), capacitance (C), transistors and gates, interconnected in various complex structures. Those analyses often need to strike a balance between accuracy and speed as those analyses need to be carried out many times throughout the entire IC design cycles. With the advancement of AI, researchers also start to explore news ways in leveraging AI to improve those analyses. This paper focuses on one of the most important analyses, timing analysis for interconnects. Since IC interconnects can be represented as an RC-tree, a specialized graph as tree, we design a novel tree-based graph neural network, SyncTREE, to speed up the timing analysis by incorporating both the structural and physical properties of electronic circuits. Our major innovations include (1) a two-pass message-passing (bottom-up and top-down) for graph embedding, (2) a tree contrastive loss to guide learning, and (3) a closed formular-based approach to conduct fast timing. Our experiments show that, compared to conventional GNN models, SyncTREE achieves the best timing prediction in terms of both delays and slews, all in reference to the industry golden numerical analyses results on real IC design data.
Electronic design automation (EDA) tools are indispensable for designing today's complex integrated circuits (ICs) that can have multi-billion transistors (switches) and logic gates. EDA tools can be
*Corresponding authors: Hussam Amrouch (amrouch@tum.de), Jinjun Xiong (jinjun@buffalo.edu)
Figure 1: Timing analysis for circuit interconnects.
classified as two categories: the analysis tools that help to evaluate an IC design's quality metrics (e.g., timing performance, signal integrity, and power efficiency), and the optimization tools that use the metrics to guide the various design optimization choices such as logic synthesis, transistor sizing, gate placement, and interconnect layout optimization. Those analyses are often based on physical properties of electronic circuits, such as the Kirchhoff's Current Law (KCL), Kirchhoff Voltage Law (KVL), and electronic components constitutive equations (i.e., the voltage-current characteristics of electronic components such as resistance R, capacitance C, and transistors).
To obtain the most accurate (and golden) analysis results, the industry de facto tool is SPICE ("Simulation Program with Integrated Circuit Emphasis"), which solves a set of differential algebraic equations (DAEs) derived from KCL and KVL via numerical integration methods. Though SPICE simulation is accurate, it is also notoriously slow and not applicable to large-scale ICs' design optimization. Therefore, most EDA analyses tools resort to some mathematical approximation techniques while still follow the physical principles to solve the DAEs in order to strike a balance between accuracy and speed, because those analyses need to be carried out many times throughout the entire cycle of IC design optimization. With the advancement of AI, researchers also start to explore news ways in leveraging AI to improve those traditionally physics-based analysis methods [1].
This paper focuses on one of the most important analyses, timing analysis, for IC interconnects (a.k.a. wires) as shown in Figure 1. For a modern IC design, there are billions of on-chip interconnects that connect billions of transistors and gates, and the total wire length can be as long as hundreds of kilometers. The interconnects help to propagate electronic signals (as represented in a voltage waveform) from a driving gate (the source) to its various downstream receiving gates (the sinks), and the interconnects can be modeled as a distributed RC-tree circuits as shown in the middle of Figure 1. In RC tree, a path is composed of all nodes and edges from the driving source (e.g. A) to a specific leaf node or sink (e.g. B and C). Path delay is defined as the time delay of a voltage waveform propagating through the path measured at the  $50\%$  of the waveform's voltage level. At the leaf node, the transition time of a rising voltage waveform from the  $10\%$  voltage level to the  $90\%$  of voltage level is defined as the waveform's slew (similarly, a falling voltage waveform's slew is defined as the transition time from  $90\%$  to  $10\%$  voltage levels). Delays capture how fast the voltage waveform propagates from the source to the sinks, and slews reflect how well the propagated voltage waveform looks like at the sinks, that is why both delays and slews metrics are of utmost importance in IC design.
Since tree is a specialized graph, it seems logical to apply graph neural networks (GNNs) [2] to learn a fast timing analysis model from many of known interconnect RC-trees. However, a direct application of modern GNN models will not work well because they do not take into the physical properties of the underlying electronic circuits. For example, as mentioned in prior works [3, 4, 5, 6], most message-passing GNNs cannot retrieve global context dependencies because of the over-smoothing effect. To address this problem, in this paper, we propose a novel tree-based GNN model based on a directed two-pass message-passing mechanism, SyncTREE as shown in Figure 2 to speed up the interconnect RC-tree's timing analysis.
In summary, our major contributions are:
- We formulate a novel GNN model for circuit timing prediction which can provide powerful expressiveness by incorporating global and subbranch dependencies during message passing. To better guide the graph learning process, we further design a new contrastive loss based on Circle loss [7], called Tree Contrastive (TC) loss, to take advantage of the monotone properties of timing delays along an RC path. Importantly, to our best knowledge, this is the first closed-form solution of leveraging GNNs to perform circuit timing analysis.
Figure 2: Architecture of our SyncTREE model.
- We create two new interconnect data sets and evaluate our model on them, one synthetically generated and one extracted from the real RISC-V IC designs. According to the experiment results, SyncTREE achieves the best accuracy for both delay and slew prediction when compared with modern GNN models, such DeepGCN[8], GraphTrans[9], and NeuralTree[10]. Furthermore, the experiments validate the advantages of TC loss in guiding our model's learning over Circle loss.
- We analyze the time complexity of SPICE and our model. From the time cost measurement results of the SPICE simulation and our model's inference, it is shown that our model can effectively speed up the timing process with little time increase along with circuit size. Besides, We further evaluate our model's expressiveness by transfer learning experiments on delay and slew prediction tasks. The results show that the representation learned from delay prediction task can effectively transfer to slew prediction task with better performance.
GNNs in IC Design. Recently, AI for EDA has become a cutting-edge research topic. Among learning-based IC design methodologies, GNNs exhibit great potential in enhancing IC design workflows. Some recent researches leverage GNNs (1) to speed up optimization in physical design [11], layout [12], placement[13], and analog design [14]; (2) to improve the prediction analysis of net length [15], timing [16], and routing congestion [17]; and (3) to even produce configuration and guidance in floorplanning [18] and 3D integration [19].
Other Related GNNs. Despite the success of GNNs, most of them are limited to capture long-range context information. Inspired by residual neural networks (ResNets), Li et al. [8] proposed the DeepGCN model to gain a large receptive field with skip connections and dilated convolutions. This allows for the training of very deep GCNs with alleviated gradient vanishing issues. Recently, motivated by the great success of Transformers, Wu et al. [9] proposed the GraphTrans model to learn representations of long-range context. It adds a CLS token to aggregate all pairwise interactions to obtain global graph embedding, and it achieves state-of-the-art results on several graph classification tasks. Although deep GNNs can broaden the receptive fields, it may also lead to node representations that are indistinguishable across the entire graph. Some other approaches are proposed is to enhance node features by incorporating rich information of important subgraph structures. For instance, Neural Tree [10] is a special GNN model that incorporates sub-structural information into feature embeddings by performing message passing on a pre-processed H-tree instead of the original input graph. The H-tree is hierarchically organized from top to down by different subgraphs which are decomposed by the junction-tree algorithm. The benefit of H-tree is that it connects each node to
its sub-graphs which makes the Neural Tree model can learn from substructures. However, it will confuse the global structural information because of its decomposition-based methodology.
Graph Contrastive Learning. Being popular in the visual representation learning field, contrastive learning [20] aims to learn discriminative representations for positive and negative samples constructed by the data augmentation process. For graph representation, recent works [21, 22, 23, 24] try to perform contrastive learning to capture rich structure information and make features in agreement with expected transformations in a supervised manner.
To obtain the golden results accurately for an electronic circuit, the industrial de facto simulation tool is SPICE [25], through modified nodal analysis (MNA) [26], which builds a system of algebraic differential equations (DAEs). For an arbitrary RC circuit, the DAEs can be constructed by following the physical constraints of KCL, KVL, and the electronic components' branch constitutive equations (i.e., the voltage-current characteristics of resistance R, capacitance C, and voltage sources E), and it can be written as
$$
\left[ \begin{array}{c c} A _ {C} C A _ {C} ^ {\prime} & 0 \\ 0 & 0 \end{array} \right] \frac {d}{d t} \left[ \begin{array}{l} e \\ i _ {V} \end{array} \right] + \left[ \begin{array}{c c} A _ {R} G A _ {R} ^ {\prime} & A _ {V} \\ A _ {V} ^ {\prime} & 0 \end{array} \right] \left[ \begin{array}{l} e \\ i _ {V} \end{array} \right] = \left[ \begin{array}{l} 0 \\ E \end{array} \right] \tag {1}
$$
where  $C$  is the diagonal matrix containing all capacitance;  $G$  is the diagonal matrix of all conductance (i.e., the inverse of the resistance  $R$ );  $E$  is the vector of voltage sources' values;  $A_{C}$ ,  $A_{R}$ , and  $A_{V}$  are the incidence matrices built from element types of capacitance  $C$ , resistance  $R$ , and voltage source  $V$ , respectively;  $i_{V}$  is the unknown current vector through voltage sources, and  $e$  is the vector of unknown node voltages. The propagation delays and slews of voltage waveforms at all sink nodes can be obtained by solving the above DAEs via numerical integration methods. SPICE usually applies a time-discretization scheme (e.g., the backward Euler method) to solve the DAEs at every time step numerically, leading to particularly intensive computations and long runtime.
Problem. Path delays and slews are two critical timing metrics that guide almost every optimization step in an IC design flow, and how to compute them accurately and quickly is of utmost importance. Accurate circuit simulation through SPICE is possible, but it would take very long time. Moreover, as the IC design process is iterative with incremental changes to the interconnect's RC parameters, SPICE simulation cannot take advantage of such incremental updates. Our objective is to design a fast and accurate timing prediction model by leveraging the powerful representation ability of GNNs.
4 Physics-informed SyncTREE for Interconnect Timing Analysis
Since electronic circuits have complicated dynamics, it is a big challenge for a GNN model to represent rich structural information at different levels. The key idea of our SyncTREE model is to construct a two-pass message-passing for our tree-based graph representation.
First, we define an undirected graph  $\mathcal{G} = (\mathcal{V},\mathcal{E})$  corresponding to an RC tree extracted from IC interconnects, where each vertex represents a grounded capacitance, and each edge represents the resistance between two vertices. Instead of relying on the undirected graph representation for the RC-tree  $\mathcal{G} = (\mathcal{V},\mathcal{E})$ , we derive two directed graph representations,  $\mathcal{G}_{bu} = (\mathcal{V},\mathcal{E}_{bu})$  and  $\mathcal{G}_{td} = (\mathcal{V},\mathcal{E}_{td})$  with  $\mathcal{G}_{bu}$  representing the directed bottom-up tree while  $\mathcal{G}_{td}$  representing the directed top-down tree. For simplicity, we will use the general graph representation  $\mathcal{G} = (\mathcal{V},\mathcal{E})$  to discuss the common features between  $\mathcal{G}_{bu}$  and  $\mathcal{G}_{td}$  whenever there is no ambiguity. Otherwise, we will denote them explicitly. Given a graph  $\mathcal{G} = (\mathcal{V},\mathcal{E})$ , the feature of each node  $i\in \mathcal{V}$  in hidden layer  $l$  is represented by vector  $h_i^l\in R^{d_l}$ . We adopt the general graph kernel from Graph Attention Networks (GAT) [27] as our message-passing backbone. We will leave the exploration of other GNN message-passing mechanisms as our future work.
4.1 SyncTREE Overview
Figure 2 shows the overall model architecture (or algorithm) for SyncTREE. The RC-Tree circuit is represented as two separate directed graphs,  $\mathcal{G}_{bu}$  for the directed bottom-up tree and  $\mathcal{G}_{td}$  for the directed top-down tree. We first perform GAT like message-passing on the bottom-up graph  $\mathcal{G}_{bu}$  by
following the bottom-up directed edges. In doing so, we propagate information from the leaf nodes, through the sub-branches, toward the source nodes. The level of influences will depend on the depth of layers  $l_{L}$  as used in GAT. At the end, every node's embedding is noted as  $h_{bu}^{l_{L}}$ . We then perform GAT like message-passing on the top-down graph  $\mathcal{G}_{td}$  by following the top-down directed edges. Different from the bottom-up pass, we first copy the hidden feature of  $h_{bu}^{l_{L}}$  from the bottom-up graph  $\mathcal{G}_{bu}$  to initialize the corresponding nodes' features in the top-down graph  $\mathcal{G}_{td}$ . After that, we apply GAT like message-passing top-down on the top-down graph  $\mathcal{G}_{td}$  with its own depth of layers  $l_{L}$ . At the end, we treat the node representations of the last layer in  $\mathcal{G}_{td}$  as the final GNN embeddings and then feed it into the linear layer for readout.
4.2 Details of Message Passing in SyncTREE
Inside basic Message-passing layers, we use GAT as the basic block to collect information. Since we treat the node capacitance as the node's attribute and wire resistance as the edge attribute in RC tree graphs, in order to preserve wire resistance information in embeddings, we modify the aggregation mechanism inside GAT to linearly combine node feature  $h_j$  and edge feature  $e_{ij}$  with normalized attention coefficients  $\alpha_{ij}$  as the final output features for every node. The update of the node state after a single hidden layer follows:
$$
\alpha_ {i j} = \frac {\exp (\text {L e a k y R e L U} \left(\mathrm {a} ^ {T} \left[ \Theta h _ {i} \right| | \Theta h _ {j} | | \Theta_ {e} e _ {i j} ]\right))}{\sum_ {k \in \mathcal {N} (i) \cup i} \exp (\text {L e a k y R e L U} \left(\mathrm {a} ^ {T} \left[ \Theta h _ {i} \right| | \Theta h _ {k} | | \Theta_ {e} e _ {i k} ]\right))} \tag {2}
$$
$$
h _ {i} ^ {l + 1} = \mathbf {A G G} ^ {l} \left(\left\{\left(h _ {j} ^ {l}, e _ {i j}\right) \mid j \in \mathcal {N} (i) \cup i \right\}\right) = \sigma \left(\sum_ {j \in \mathcal {N} (i) \cup i} \alpha_ {i j} \left(\Theta h _ {j} ^ {l} + \Theta_ {e} e _ {i j}\right)\right) \tag {3}
$$
where  $\Theta$  and  $\Theta_{e}$  are separately applied to every node and every edge as linear transformations,  $||$  represents concatenation,  $T$  is transpose operation, and  $\mathcal{N}(i)\subseteq \mathcal{V}$  is the neighbor node set of  $i$ .
In SyncTREE, we respectively applied  $\mathrm{GAT}_{bu}$  and  $\mathrm{GAT}_{td}$  to perform message passing on the bottom-up tree  $\mathcal{G}_{bu}$  and the top-down tree  $\mathcal{G}_{td}$ , respectively.
The feature representation in  $\mathcal{G}_{bu}$  is updated by  $h_{i,bu}^{l + 1} = \mathrm{AGG}_{bu}^{l}(\{(h_{j,bu}^{l},e_{ij,bu})|j\in \mathcal{N}(i)\cup i\})$  When we start the top-down message-passing, we use the aggregation output of the final layer in  $\mathcal{G}_{bu}$  to initialize the node features in  $\mathcal{G}_{td}$
$$
h _ {i, t d} ^ {0} = h _ {i, b u} ^ {L} \tag {4}
$$
For every convolutional layer in  $\mathcal{G}_{td}$ , the node hidden states will be updated by two parts: node representations updated from the previous layer of  $\mathcal{G}_{td}$  itself and the copied final embedding from  $\mathcal{G}_{bu}$ . It can be expressed as:
$$
h _ {i, t d} ^ {l + 1} = \mathbf {A G G} _ {t d} ^ {l} \left(\left\{\left(h _ {j, t d} ^ {l}, e _ {i j, t d}\right) \mid j \in \mathcal {N} (i) \cup i \right\}\right) + h _ {i, b u} ^ {L} \tag {5}
$$
Since  $\mathrm{AGG}_{bu}$  and  $\mathrm{AGG}_{td}$  are performed on two directed graphs, SyncTREE can effectively avoid over-smoothing and preserve variance among nodes. In our case, we aim at predicting path timing, thus we use the final predictions at leaf nodes (sinks) by applying a mask to calculate the loss function.
4.3 Tree Contrastive Loss
It is a well-known fact that the voltage along an RC path is a monotonic function of distance [28]. For a target node, in its RC path to the root node, its timing would be similar to its nearest node and have bigger differences with further nodes. Inspired by contrastive learning, we introduce tree contrastive (TContrast) loss to enhance node embedding quality by optimizing the pair similarity of node representations. The idea behind contrastive learning is to pull positive samples close to the target node and push the negative samples away.
At every batch during training, we randomly sample a set of target leaf nodes with size  $B$ . For any target leaf node  $i \in B$ , we get a node embedding set  $S_{i}$  by picking the final representations  $h_{i,td}^{L}$  of positive and negative nodes from  $\mathcal{V}$ . To determine positive and negative samples of a target leaf node, we set a hops hyperparameter. On the RC path from the target node to the source node, we treat the nodes within the hops to the target as positive samples and the others as negative samples.
It should contain  $M$  positive samples and  $N$  negative samples, which means that there will be  $M$  pair of within-class similarity  $s_p$  and  $N$  pairs of between-class similarities  $s_n$ . We apply Euclidean distance to measure similarity as follows:
$$
s _ {i j} = \frac {1}{1 + \alpha * \left\| h _ {i , t d} ^ {L} - h _ {j , t d} ^ {L} \right\| _ {2}}, h _ {j, t d} ^ {L} \in \mathcal {S} _ {i} \tag {6}
$$
where  $\alpha$  is used to avoid  $s_{ij}$  being too small. For pair similarity optimization, it is natural to maximize within-class similarity  $s_p$  and minimize between-class similarity  $s_n$ . A novel optimization manner of Circle loss is to reduce  $(\alpha_n s_n - \alpha_p s_p)$ , where the  $\alpha_n$  and  $\alpha_p$  are independent weight factors. The intuition behind it is different similarity scores should have different penalty strengths. For the Circle loss, it aims to optimize  $s_p \to 1$  and  $s_n \to 0$ , and use  $m$  the radius of the circle decision boundary [7]. However, it's unfair for the Circle loss to use the same relaxation strength  $m$  to  $s_p$  and  $s_n$ . In TContrast loss, we design a more flexible decision boundary based on the priori path resistance of samples. For each  $(s_p^k, s_n^q)$  pair, we define a coefficient  $r_d^{k,q}$  of path resistances between positive node  $k$  to target  $i$  and negative node  $q$  to target  $i$  as follows:
$$
r _ {d} ^ {k, q} = \frac {\sum_ {j = i} ^ {k} R _ {j}}{\sum_ {j = i} ^ {q} R _ {j}} \tag {7}
$$
where the numerator/denominator is the sum of resistance in RC path  $i$  to  $k / q$ . Instead of only using similarity score to amplify the gradient to  $s_p$  and  $s_n$ , we separately set distance-adaptive scaling factors based on path resistance for  $\alpha_{n}$  and  $\alpha_{p}$  by:
$$
\alpha_ {p} ^ {k, q} = e ^ {r _ {d} ^ {k, q}} \left[ \left(1 + m - s _ {p} ^ {k}\right) \right] _ {+}, \alpha_ {n} ^ {k, q} = e ^ {1 - r _ {d} ^ {k, q}} \left[ \left(s _ {n} ^ {q} + m\right) \right] _ {+} \tag {8}
$$
Following the setting in original Circle loss, we can deduce the TContrast loss of node  $i$  as:
$$
l _ {i} ^ {T C} = \log \left[ 1 + \sum_ {k} \sum_ {q} \exp \left(\gamma \left(\alpha_ {n} ^ {k, q} \left(s _ {n} ^ {q} - m\right) - \alpha_ {p} ^ {k, q} \left(s _ {p} ^ {k} - (1 - m)\right)\right)\right) \right] \tag {9}
$$
Since the decision boundary is achieved at  $\alpha_{n}(s_{n} - m) - \alpha_{p}(s_{p} - (1 - m)) = 0$ , combined with (7), the decision boundary in TContrast loss is given by:
$$
\frac {s _ {n}}{e ^ {r _ {d}}} + \frac {\left(s _ {p} - 1\right) ^ {2}}{e ^ {1 - r _ {d}}} = \frac {m ^ {2}}{e ^ {r _ {d}}} + \frac {m ^ {2}}{e ^ {1 - r _ {d}}} \tag {10}
$$
It shows that the decision boundary of TContrast loss is an ellipse arc, where the center is  $(0,1)$ , and the lengths of the semi-axis along  $s_n$  and  $s_p$  are  $m\cdot \sqrt{1 + e^{2r_d - 1}}$  and  $m\cdot \sqrt{1 + e^{1 - 2r_d}}$ . So TContrast loss expects  $s_n < m\cdot \sqrt{1 + e^{2r_d - 1}}$  and  $s_p > 1 - m\cdot \sqrt{1 + e^{1 - 2r_d}}$ . The intuitive interpretation is that we set different relaxations to  $s_n$  and  $s_p$  according to  $r_d$ . If  $r_d$  increases, it will have a more strict margin for  $s_p$  and a looser margin for  $s_n$ , which means we emphasize on improving  $s_p$  when the negative nodes close to target.
In TContrast loss  $\mathcal{L}_{TC}$ , We formulate different penalty strengths according to the similarity score and relative distances of node pairs. We serve it as a regularizer in our final objective loss function, which also includes L1-loss  $\mathcal{L}_{l1}$  of true timing and predicted timing:
$$
\mathcal {L} _ {\text {f i n a l}} = \mathcal {L} _ {l 1} + \mathcal {L} _ {T C} = \mathcal {L} _ {l 1} + \lambda \frac {1}{B} \sum_ {i = 1} ^ {B} l _ {i} ^ {T C} \tag {11}
$$
In this section, we first explain the datasets, parameter settings, and baselines of our experiments, then demonstrate the advantages of our method in the IC timing prediction through the experiments.
Prediction Task. Given the RC trees extracted from IC interconnections and the driving voltage at the input node, we aim to predict the pin-to-pin timing including delays and slews. We perform the prediction tasks on two benchmark datasets, which are summarized in Table 1, to comprehensively evaluate the performance of our SyncTREE model. Given SPICE timing results as golden, we use
MAE as the evaluation metric for both delay and slew predictions. All experiments in this paper are implemented with PyTorch 1.13.1 and PyTorch Geometric 2.2.0 frameworks and executed on a Ubuntu server equipped with Intel Xeon Gold 6330 CPU with 56 cores/2 threads running at  $2.0\mathrm{GHz}$ . The reference SPICE simulations are carried out with the commercial Synopsys HSPICE simulator on an AMD Ryzen 3950X with 16 cores/32 threads at  $3.5\mathrm{GHz}$ . (Code and datasets are available at https://github.com/xlab-ub/SyncTree).