SyncTREE: Fast Timing Analysis for Integrated Circuit Design through a Physics-informed Tree-based Graph Neural Network
Authors: Yuting Hu
Jiajie Li
Florian Klemme
Gi-Joon Nam
Tengfei Ma
Hussam Amrouch
Jinjun Xiong
Venue: NeurIPS 2023 poster (2023)
Keywords: Graph Neural Networks
Integrated Circuits
Circuit Timing Analysis
Physics-guided Deep Learning

Abstract: Nowadays integrated circuits (ICs) are underpinning all major information technology innovations including the current trends of artificial intelligence (AI). Modern IC designs often involve analyses of complex phenomena (such as timing, noise, and power etc.) for tens of billions of electronic components, like resistance (R), capacitance (C), transistors and gates, interconnected in various complex structures. Those analyses often need to strike a balance between accuracy and speed as those analyses need to be carried out many times throughout the entire IC design cycles. With the advancement of AI, researchers also start to explore news ways in leveraging AI to improve those analyses. This paper focuses on one of the most important analyses, timing analysis for interconnects. Since IC interconnects can be represented as an RC-tree, a specialized graph as tree, we design a novel tree-based graph neural network, SyncTREE, to speed up the timing analysis by incorporating both the structural and physical properties of electronic circuits. Our major innovations include (1) a two-pass message-passing (bottom-up and top-down) for graph embedding, (2) a tree contrastive loss to guide learning, and (3) a closed formular-based approach to conduct fast timing.  Our experiments show that, compared to conventional GNN models, SyncTREE achieves the best timing prediction in terms of both delays and slews, all in reference to the industry golden numerical analyses results on real IC design data.

SyncTREE: Fast Timing Analysis for Integrated Circuit Design through a Physics-informed Tree-based Graph Neural Network"
Yuting Hu University at Buffalo Buffalo, NY, USA yhu54@buffalo.. edu
Jiajie Li University Buffalo Buffalo, NY, USA j11433@buffalo. edu
Florian Klemme
Gi-Joon Nam IBM Research Yorktown Heights, NY, USA gnam@us ibm. com
University of Stuttgart Stuttgart, Germany
klemme@iti.um uni uni.stutttgrrr.dde
Tengfei Ma
Hussam Amrouch
Jinjun Xiong University at Buffalo Buffalo, NY, USA jinjunbbuffan edu
Stony Brook University Stony Brook, NY, USA
Technical University of Munich
München, Germany
tengfei www.tonybrookkocooooooo edu
amr amrouch@tum.co
Abstract
Nowadays integrated circuits (ICs) are underpinning all major information technol- ogy innovations including the current trends of artificial intelligence (AI). Modern IC designs often involve analyses of complex phenomena (such as timing, noise, and power etc.) for of billions electronic components, like resistance (R), capacitance (C), transistors and gates, interconnected in various complex structures. Those analyses often need to strike balance between accuracy and speed as those analyses need to be carried out many times throughout the entire IC design cycles. With the advancement
AI, researchers also start to explore news ways in leveraging AI to improve those analyses. This paper focuses on one of the most important analyses, timing analysis for interconnects. Since interconnects can be represented as RC-tree, specialized graph tree, we design a novel tree-based graph neural network, SynETREE, to speed up the timing analysis by incorporating both the structural and physical properties of electronic circuits. Our major innovations include (1) a two-pass message- pressage-passing (bottom-up and top-down) for graph embedding. (2) tree contrastive loss to guide learning, and (3) a closed Cormular-basee approach to conduct fast timing. Our experiments show that, compared to conventional GNN models, SyncTREE achieves the best timing prediction in terms of both delays and slews, all reference to the industry golden numerical analyses results real IC design data.
1 Introduction
Electronic design automation (EDA) tools are indispensable for designing today's complex integrated circuits (ICs) that have multi- billi--illionn transistors (switches) and logic gates. EDA tools can be
"Correspondng authors: Hussam Amrouch (amrouch @tum.de), Jinjun Xiong (jinjun @ butfalo.edu)
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

0.7
Ww
Supply Supply vottgge [V]
0.6
Plo Ir
Ptn
Interconnedl
64
6.5
6.2
delay (A- - Cy.
interpornect
6.1
slow
Interoonnect Design
RC Tree Circuit
0.56
0.57
0,50
SPICE simulation time [ns]
Figure 1: Timing analysis for circuit interconnects.
classified as two categories: the analysis tools that help to evaluate IC design's 's quality metrics (e.g., timing performance, signal integrity, and power efficiency), and the optimization tools that use the metries guide the various design optimization choices such as logic synthesis, transistor sizing, gate placement, and interconnect layout optimization. Those analyses are often based on physical properties of electronic circuits, such as the Kirchhoff's Current Law (KCL), Kirchhoff Voltage Law (KVL), and electronic components constitutive equations (i.e., the voltage-current characteristics of electronic components such resistance R, capacitance C, and transistors).
To obtain the most accurate (and golden) analysis results, the industry de factor tool is SPICE ("Simulation Program with Integrated Circuit Emphasis"), which solves a set of differential algebraic equations (DAEs) derived from KCL. and KVL via numerical integration methods. Though SPICE simulation is accurate, it is also notoriously slow and not applicable to large-scale ICs' design optimization. Therefore, most EDA analyses tools resort to some mathematical approximation techniques while still follow the physical principles to solve the DAEs in order to strike a balance between accuracy and speed, because those analyses need to be carried out many times throughout the entire eycle of IC design optimization. With the advancement of AI, researchers also start to explore news ways in leveraging Al to improve those traditionally physics- based analysis methods [..
This paper focuses on one of the most important analyses, timing analysis, for IC interconnects (a.k.a. wires) as shown in Figure D For a modern IC design, there are billions of on-chip interconnects that connect billions of transistors and gates, and the total wire length can be as long as hundreds of kilometers. The interconnects help to propagate electronic signals (as represented in a voltage waveform) from driving gate (the source) to its various downstream receiving gates (the sinks), and the interconnects can be modeled as a distributed RC-tree circuits as shown in the middle of Figure T In RC tree, a path is composed of all nodes and edges from the driving source (e.g. A) to a specific leaf node or sink (e.g. and C). Path delay is defined as the time delay of a voltage waveform propagating through the path measured the 50% of the waveform's 's voltage level. At the leaf node, the transition time of a rising voltage waveform from the 10% voltage level to the 90% of voltage level is defined the waveform's 's slew (similarly, falling voltage waveform's 's slew is defined as the transition time from 90% to 10% voltage levels). Delays capture how fast the voltage waveform propagates from the source the sinks, and slews reflect how well the propagated voltage waveform looks like at the sinks, that is why both delays and slews metrics are of utmost importance in IC design.
Since tree is a specialized graph, it seems logical to apply graph neural networks (GNNs) 2 to learn a fast timing analysis model from many of known interconnect RC-trees. However, direct application of modern GNN models will not work well because they do not take into the physical properties of the underlying electronic circuits. For example, mentioned in prior works B 4 message-passing GNNs cannot retrieve global context dependencies because of the over-smoothing effect. To address this problem, in this paper, we propose novel tree-based GNN model based on a directed two-pass message-passing mechanism, SyneTREE as shown in Figure to speed up the interconnect RC-tree's 's timing analysis.
most
In summary, our major contributions are:
We formulate a novel GNN model for circuit timing prediction which can provide powerful expressiveness by incorporating global and subbranch dependencies during message passing. To better guide the graph learning process, we further design new contrastive loss based on Circle loss (7) called Tree Contrastive (TC) loss, to take advantage of the monotone properties of timing delays along an RC path. Importantly, to our best knowledge, this the first closed- solution of leveraging GNNs to perform circuit timing analysis.

---

Embeddings A
Embeddings R
AGG
AGG
Readout
Lincar
Init Nodes
Mask
Tree
Wire Delay/Slew
BottomUp Propagation
Topl TowDown
Propagation
Figure 2: Architecture of our SyncTREE, model.
We create two new interconnect data sets and evaluate our model on them, one synthetically generated and extracted from the real RISC-V IC designs. According the experiment results, SyncTREE achieves the best accuracy for both delay and slew prediction when com- pared with modern GNN models, such DeepGCN(8), GraphTrans and NeuralTree[[O) Furthermore, the experiments validate the advantages of TC loss in guiding our model's learning over Circle loss.
We analyze the time complexity of SPICE and our model. From the time cost measurement results of the SPICE simulation and our model's inference, it is shown that our model can effectively speed up the timing process with little time increase along with circuit size. Besides, We further evaluate our model's expressiveness by transfer learning experiments on delay and slew prediction tasks. The results show that the representation learned from delay prediction task can effectively transfer to slew prediction task with better performance.
2
Related Work
GNNs in IC Design. Recently, AT for EDA has become a cutting-edse research topic. Among learning-based IC design methodologies, GNNs exhibit great potential in enhancing IC design workflows. Some recent researches leverage GNNs (1) to speed up optimization in physical design [T]] layout [I2], placement[13] and analog design [4] (2) improve the prediction analysis of
net length [1S], timing (16), and routing congestion 7 and (3) to even produce configuration and
guidance in floorplanning (18) and 3D integration 19
Despite the success of GNNs, most of them are limited capture long- range context information. Inspired by residual neural networks (ResNets), Li et al. [ proposed the DeepGCN model to gain large receptive field with skip connections and dilated convolutions. This allows for the training of very deep GCNs with alleviated gradient vanishing issues. Recently, motivated by the great success of Transformers, Wu al. 9 proposed the GraphTrans model to learn representations of long-range context. It adds a CLS token to aggregate all pairwise interactions to obtain global graph embedding, and it achieves oftee-of-the-att results on several graph classification tasks. Although deep GNNs can broaden the receptive fields, it may also lead to node representations that are indistinguishable across the entire graph. Some other approaches are proposed to enhance node features by incorporating rich information of important subgraph structures. For instance, Neural Tree [10] is a special GNN model that incorporates sub-strie sub-structural information into feature embeddings by performing message passing on pre-processed H-tree instead of the original input graph. The H-tree hierarchically organized from top to down by different subgraphs which are decomposed by the junction-tree algorithm. The benefit of H-tree is that it connects each node to
Other Related GNNs.

---

its sub-graphi which makes the Neural Tree model can learn from substructures. However, it will confuse the global structural information because of its decomposition -based methodology.
Graph Contrastive Learning. Being popular in the visual representation learning field, contrastive learning (20] aims to learn discriminative representations for positive and negative samples con-
structed by the data augmentation process. For graph representation, recent works [2] 22 23 24 try
to perform contrastive learning to capture rich structure information and make features in agreement with expected transformations in supervised manner.
3 Problem Statement
To obtain the golden results accurately for an electronic circuit, the industrial de facto simulation tool is SPICE [25], through modified nodal analysis (MNA) [26]. which builds system of algebraic differential equations (DAEs). For an arbitrary RC circuit, the DAES can be constructed by following the physical constraints of KCL, KVL, and the electronic components' branch constitutive equations (i.e., the voltage-current characteristics of resistance R, capacitance C, and voltage sources E), and can be written
AcCA'
d 0 dt
ARGA'R Avv
Av
0 E
+
(I)
ivv
iv
where C' is the diagonal matrix containing all capacitance; (i.e., the inverse of the resistance R); E is the vector of voltage sources' values; Ac, An, and Av the incidence matrices built from element types of capacitance C, resistance R, and voltage source V. respectively; iv is the unknown current vector through voltage sources, and e is the vector of unknown node voltages. The propagation delays and slews of voltage waveforms at all sink nodes can be obtained by solving the above DAEs via numerical integration methods. SPICE usually applies time-discretization scheme (e.g.. the backward Fuler method) to solve the DAES at every time step numerically, leading to particularly intensive computations and long runtime.
the diagonal matrix of all conductance
Problem. Path delays and slews are two critical timing metrics that guide almost every optimization step in an IC design flow, and how to compute them accurately and quickly is of utmost importance. Accurate circuit simulation through SPICE is possible, but it would take very long time. Moreover, as the IC design process is iterative with incremental changes the interconnect's s RC parameters, SPICE simulation cannot take advantage of such incremental updates. Our objective is to design a fast and accurate timing prediction model by leveraging the powerful representation ability of GNNs.
4 Physics-informed SyncTREE for Interconnect Timing Analysis
Since electronic circuits have complicated dynamics, it is a big challenge for a GNN model to represent rich structural information different levels. The key idea of our SyncTREE model is to peessag-ppssiing construct a two-pass message- passing for our tree-based graph representation.
First, define undirected graph G (V,E) corresponding to an RC tree extracted from IC interconnects, where each vertex represents a grounded capacitance, and each edge represents the resistance between vertices. Instead of relying the undirected graph representation for the RC- tree G = (V, E), we derive two directed graph representations, Gon = (V. Ebu) and Gee = (V, Era). with Go. representing the directed bottom-up tree while Gia representing the directed top-down tree. For simplicity, we will use the general graph representation G = (V, E) to discuss the common features between Gon and Ged whenever there is no ambiguity. Otherwise, we will denote them explicitly. Given a graph G = (V, E), the feature of each node E V in hidden layer is represented by vector h² e Rd. We adopt the general graph kernel from Graph Attention Networks (GAT) [27 message- passing backbone. We will leave the exploration of other GNN message-passing mechanisms as our future work.
4.1 SynCTREEF Overview
Figure shows the overall model architecture (or algorithm) for SyncTREE. The RC-Tree circuit represented as two separate directed graphs, Gon for the directed bottom-up tree and Gaa for the directed top-down tree. We first perform GAT like message-passing the bottom-up graph Go.. by

---

following the bottom-up up directed edges. In doing so, we propagate information from the leaf nodes, through the sub-branches, toward the source nodes. The level of influences will depend on the depth of layers I used in GAT. At the end, every node's embedding noted as h We then perform GAT like message-passing passing on the top-down graph Gtd by following the top-down directed edges. Different from the bottom-up pass, we first copy the hidden feature of h/ from the bottom-up graph 9hu to initialize the corresponding nodes' features in the top-down graph Gta- After that, we apply GAT like message-passing top-down on the top-down graph Gea with its own depth of layers l.- At the end, we treat the node representations of the last layer in Gia as the final GNN embeddings and then feed it into the linear layer for readout.
4.2 Details of Message Passing in SyncTREE
Inside basic Message- passggng layers, we use GAT as the basic block to collect information. Since we treat the node capacitance as the node's 's attribute and wire resistance as the edge attribute in RC tree graphs, in order to preserve wire resistance information in embeddings, we modify the aggregation mechanism inside GAT to linearly combine node feature h, and edge feature ej with normalized attention coefficients og as the final output features for every node. The update of the node state after single hidden layer follows:
exp(LeakyRed.//a][0)))) ebp_aaabkee///////////////////))
aij EKEN(OLL exp(Leaky |1hn (0,,][9,eal))
(2)
h1t1 = AGG (I(h), eq)li N(i) =a
Σ (n)( + Oeer))
(3)
JEN((iUU
where and O₂ are separately applied to every node and every edge as linear transformations, represents concatenation, T is transpose operation, and (i)) C V is the neighbor node set of i.
In SyncTREF, we respectively applied GAT bu and GAT to perform message passing on the bottom-up tree Gbn. and the top-down tree Gid, respectively.
The feature representation in Gou is updated by highu AGG!.. (1(h)) b.b enj, E (N()
1+1
When we start the top-down message-piin pesssagepaasing, we use the aggregation output of the final layer in Gin to initialize the node features in Gea:
1 = h
(4)
For every convolutional layer in 9ed, the node hidden states will be updated by two parts: node representations updated from the previous layer of Gea itself and the copied final embedding from Gbu. It can be expressed as:
h h+1 AGG eyyalli E N(I) + h4
Since AGG and AGG are performed on two directed graphs, SyncTREE effectively avoid over-smoothing and preserve variance among nodes. In our case, we aim at predicting path timing, thus use the final predictions leaf nodes (sinks) by applying mask calculate the loss function.
4.3 Tree Contrastive Loss
is a well-known fact that the voltage along an RC path is a monotonic function of distance (2 For target node, in its RC path the root node, its timing would be similar its nearest node and have bigger differences with further nodes. Inspired by contrastive learning, we introduce tree contrastive (TContrast) loss to enhance node emhedding quality by optimizing the pair similarity of node representations. The idea behind contrastive learning is to pull positive samples close to the target node and push the negative samples away.
At every batch during training, we randomly sample a set of target leaf nodes with size B. For any target leaf node E B, get node embedding set S, by picking the final representations h/a of positive and negative nodes from V. To determine positive and negative samples of target leaf node, we set hops hyperparameter. On the RC path from the target node to the source node, we treat the nodes within the hops to the target as positive samples and the others as negative samples.
5

---

It should contain M positive samples and N negative samples, which means that there will be M pair of within-class similarity 8p and N pairs of between-class similarities 8n- We apply Euclidean distance to measure similarity as follows:
Sij
Saa S
(6)
1 a
M "j,ed]
where a is used to avoid Sty being too small. For pair similarity optimization, it natural to maximize within-class similarity Sp and minimize between-class similarity Sn. A novel optimization manner of Circle loss is to reduce (ongn apsp), where the On, and ap are independent weight factors. The intuition behind it is different similarity scores should have different penalty strengths. For the Circle loss, it aims to optimize 8p - and 8n, - O, and use m the radius of the circle decision boundary 77]. However, it's unfair for the Circle loss use the same relaxation strength to 8p and 8n- In TContrast loss, we design more flexible decision boundary based on the priori path resistance of ki of path resistances between positive node
samples. For each (s s%) pair, we define a coefficient rd to target i and negative node y to target as follows:
Σ Di=i R3 Ej. R3
where the numerator/denomiiatorr is the sum of resistance in RC path i to k/g. Instead of only using similarity score to amplify the gradient Sp and Sn, we separately distance-adaptive scaling factors based on path resistance for an and Op by:
R.9 02
e [[1 + m - s an =
e el-rd [[5 + m)]+
(8)
Following the setting in original Circle loss, we can deduce the TContrast loss of node i as:
To log(1 + ΣΣ exp(>(a (s m) a (s (11 (1)))))
(9)
Since the decision boundary achieved at an (sn m) ap(sp - (1 (1-m))= 0, combined with (7), the decision boundary in TContrast loss is given by:
8p (8p 1)2
1)2
m
m
(10)
eta
cl-ru
eta
el-ra
It shows that the decision boundary of TContrast loss is ellipse are, where the center is (0, 1), and the lengths of the remi-axis along Sn and Sp m vI + - and m vI e So TContrast loss expects 8n m vI+ e2ra-I and sp that we set different relaxations to 8n and Sp according to If increases, it will have a more strict margin for Sp and looser margin for sn, which means we emphasize on improving Sp when the negative nodes close to target.
vI= el-2ra, The intuitive interpretation is
In TContrast loss Lrc, We formulate different penalty strengths according to the similarity score and relative distances of node pairs. We serve as a regularizer in our final objective loss function, which also includes L.-lloss Lu of true timing and predicted timing:
1
Litnat L1 LrC Lu
(11)
B
5 Experimental Setting
In this section, we first explain the datasets, parameter settings, and baselines of our experiments, then demonstrate the advantages of our method in the IC timing prediction through the experiments.
Prediction Task.
Given the RC trees extracted from IC interconnections and the driving voltage at the input node, we aim to predict the pin-to-pin timing including delays and slews. We perform the prediction tasks on two benchmark datasets, which are summarized in Table T to comprehensively evaluate the performance of our SyncTREE model. Given SPICE timing results as golden, use

---

MAE as the evaluation metric for both delay and slew predictions. All experiments in this paper are implemented with PyTorch 1.13.1 and PyTorch Geometric 2.2.0 frameworks and executed Ubuntu server equipped with Intel Xeon Gold 6330 CPU with 56 cores/2 threads running at 2.0GHz. The reference SPICE simulations are carried out with the commercial Synopsys HSPICE simulator on an AMD Ryzen 3950X with 16 cores/32 threads at 3.5GHz. (Code and datasets are available at http://ggthbbbooom/aaaauu///nnnt
Table 1: Statistics of Datasets
Synthetic Dataset
RISC-V V Dataset
Configuration
uniform distribution
min
average
max
Nodes
2-51 nodes
2 nodes
6.21 nodes
20 nodes
Resistance
10-2000 ohm
0.5 ohm
101.6 ohm
549.1 ohm
Capacitance
0.01-2 fF
0.00113 fF
0.192 fF
2.398 fF
Samples
4,066 circuits
414.639 circuits
The dataset consists of synthesized RC circuits with various circuit typologies, including rare interconnections. To generate these circuits, we first generate an RC tree and then simulate its corresponding artificial IC interconnects with SPICE to obtain the golden timing values. The pscudocode for generating RC trees, the workflow for data preparation, and the golden timing distribution in data samples are provided in the Appendix.
Synthesized Dataset.
RISC-V dataset. The dataset is composed of real RC circuits extracted from practical RISC-V IC designs. Our objective is to validate the effectiveness of our model in analyzing the timing of interconnects in practical IC designs.
Models and parameters. For our SynETREE model, we separately set 32 hidden dimensions on the synthetic dataset and 128 hidden dimensions on the RISC-V dataset. The TContrast loss is set with hops of 2, N of 1, in the range of [0.1, and B is 64 for the synthetic dataset and 128 for the RISC-V dataset. All the models are trained with the Adam optimizer (31 0.9, 32 0.99). The batch size is set 32 for the synthetic dataset and 256 for RISC-V dataset. The learning rate was set to 8e-4 for 4/8/16-layse 1--yer models and 4c-4/2e-4 for 32/64-layer models. All the models are trained for 10 epochs on the RISC-V V dataset and 60 epochs on the synthetic dataset.
Baselines. Considering timing analysis is unique task that depends on the structures of RC trees at different levels, we compare our method to various conventional GNNs, including GCN (2)). GAT (27), and GraphS and NeuralTree[[D [0]. For GraphTrans, to incorporate complex global information into node features, we concatenate the CLS token embedding of Transformer with node embeddings and input to MLP
[30] as well prospective GNNs such as DeepGCN(8), GraphTrans
to get the final output.
Results and Discussion
In this section, we present a comprehensive evaluation our SyncTTree model's performance com- pared to selected GNN baselines for delay and slew prediction tasks and showcase the impact of using TContrast loss function for training. Then, we discuss the computational efficiency of SyncTREE and its transfer learning capabilities.
Main Results. We present the delay and slew prediction results of different GNN models on the Synthetic and RISC-V datasets in Table 2 and Table 3 The results show that TREE outperforms the existing GNN baselines across all model depths on both synthetic and RISC-V datasets. For example, even the only four-layer SyncTREE model nearly surpasses all the other baselines with different model depths. Furthermore, our model consistently outperforms the baselines on the RISC- V dataset, highlighting its applicability to real-world scenarios. To evaluate the effectiveness of graph representations obtained by different GNN models, we visualize the final node embeddings of the converged GNNs in Figure 5 and calculate the Pearson correlation coefficient with the target delay distribution across the entire circuit graph. The results show that our model can achieve the highest correlation score compared other baselines, further validating its superiority.
7

---

Table Mean Average Error of Prediction Results in Synthetic Dataset
Delay Prediction Error (ps)
Layers
Baselines GraphSAGE
Ours SynTTREE+TC
GCN
GAT
DeepGCN
GraphTrans
NTREE
SyncTREE
SyncTREE+C
12.675
11.192
9.833
9.231
9.077
8.745
8.622
8
13.037
9.834
9.798
9.193
9.283
9.432
6.631
6.521
6.590
9.438
3.775
32
13.353
6,942
10.278
9.363
9,518
8.217
3.424
3.354
3.370
13.782
13.356
10.232
9.178
8.859
7,022
3.556
3.451
3.523
Slew Prediction Error
Layers
Baselines
Ours SyEETREE+TC
GCN
GraphSAGE
DeepGCN
GraphTrans
NTREE
SyncTREE
SyncTREE+C
37.348
186
32,705
29.561
29.121
29.781
25.672
25.379
26. 601
39.004
35.164
33,429
29.338
29.483
29.772
20.540
20.215
20. 20.620
16
40.348
27.817
34,908
29.691
28.352
28.368
14.305
14.047
14.285
32
41.827
25.847
34.857
29.563
27 47.466
28.581
12.864
12.608
13.107
64
43.913
42.092
36,962
29.758
28.897
27.044
14.544
14:426
14.591
Table Mean Average Error of Prediction Results in RISC- dataset
Delay Prediction Error (ps)
Layers
Baselines
GCN
GAT
GraphSAGE
DeepGCN
GraphTrans
NTREE
SynCTREF
SybcTREE+TC
SyncTREE+C
0.0467
0.0424
0.0417
0.0430
0.0359
0.0422
0.0313
0.0306
0.0317
8
0.0403
0.0403
0.0395
0.0372
0.0341
0.0298
0.0195
0.0182
0.0223
16
0.0343
0.0285
0.0371
0,0453
0.0319
0.0263
0.0128
0.0114
0.0131
32
0.0321
0.0304
0.0405
0.0311
0.0338
0.0271
0.0106
0.0098
0.0109
64
0.0379
0.0389
0.0436
0.0450
0.0357
0.0314
0.0176
0.0159
0.0181
Prediction (ps)
Layers
Baselines
Ours SynTTREE+TC
GCN
GAT
GraphSAGE
DeepGCN
GraphTrans
NTREE
SyncTREE
SyncTREE+C
0.5453
0.4739
0.3319
0.1514
0.1312
0.2231
0.0-110
0.04106
0.01188
8
0.3719
0.3740
0.4726
0,1864
0.1436
0.1974
0.0265
0.0237
0.0258
0.3141
0.3257
0.1908
0.0228
32
0.3004
0.2989
0.3569
0.1190
0.1349
0.1859
0.0464
0.0428
0.0491
0.1875
Daly Koerip--rii Eriipl
Daley Enwilo
Narbor Modo Mode
Nadoo of Lechaooe
Narior of Mode
Naitor Hove
(a) on S
Percent enror on
MAE on
(d) Percent error on
Figure 3: Delay Prediction Result of SyncTRREE on Synthetic dataset (S) and RISC-V dataset (R).
Mes Katroa Grotiel
Reseiive moooni
Mode
Natto
Number of Mode
Number Mode
MAF
Percent S
MAE
Percent
Figure Slew Prediction Result of SyncTREE on Synthetic dataset (S) and RISC-V dataset (R).
Accuracy vs. circuit size and the number of sinks. To provide more insights into the performance of our method, investigate the relationship between prediction accuracy and two important factors: circuit size and the number of sinks (i.c., leaf nodes) in RC trees, As depicted in Figure 3 3land Figure our method achieves better accuracy on circuits with larger sizes and more sinks.
TContrast loss vs. Circle loss. We provide the prediction results of SyncTREE models which trained with TContrast (TC) loss and Circle (C) loss in Table 2 and Table 3 we found that Syne- TREE+TC performs better than the default I.I loss and Circle loss, highlighting the advantages of TC in optimizing similarity pairs and loss optimization convergence. To further compare the original

---

(a) Target
(h) (urs(0.94)
(c) GAT10.82)
(d) NTREE(0667)
(e) GrapTTamm(000
Figure 5: Visualization of final feature embeddings obtained by different GNNs.
1.0
1.0
> 0.6
0.4
02
5n
Sn
(a) Cirole Loss
TContrust loes
Figure 6: Visualization of the similarity distributions after convergence.
Circle loss and TContrast loss, we analyzed their convergence status the delay prediction task. Figure 6 shows the distribution of similarity pairs (s,, sp) after convergence. We observed that the similarity pairs using TContrast loss are closer to the optimization goal and have more concentrated distribution in similarity space.
Computational Efficiency. SPICE typically requires significant computational resources to model circuit behavior time-incremental manner. However, considering that RC circuits are linear systems, we can utilize linear multi-step numerical integration techniques like the Trapezoidal method 31 to solve the differrentallalgebbric equations if the form of dz at Ax inside SPICE for transient simulation. This results would be æ(t + h) (I- hA)) (a
hhA)x(t), given a circuit with nodes and a driving signal with m time steps from the source. With LU factorization being used to invert matrices, the computation complexity of SPICE is expressed as O(m (n² n² n)).
In contrast, the computational complexity of our SynE model significantly reduced, which makes it highly efficient. As it's GAT-based, there are extensive matrix operations required, and the computations can parallelized easily. Moreover, the number of layers
less than n. Thus, the time complexity of our model can be expressed as O (n- f f + (n - 1) f')), where f is the number input features, f' is the number of output features, and n - 1 the number of edges. We provide the computational efficiency comparison of SyncTREE with SPICE simulation in Figure 7 Our results indicate that SyncTREE is significantly faster than SPICE in obtaining accurate results, with the advantage increasing the circuit size grows.
Transfer Learning. We provide the results of ablation study of transfer learning of SyncTREE on both delay and slew tasks Table 4 During transfer learning, we froze the weights pre-trained on the other task and only fine-tuned the readout module the target task. The results show that the representations learned from the delay prediction task can be efficiently transferred to the slew task, resulting in better performance. The indicates that SynETREE's learned representation can be efficiently transferred between tasks, which can save time and resources by avoiding the need for additional training every target task from scratch.
Conclusion
In this paper, we propose novel GNN model to predict circuit timing with a much faster computation speed than the SPICE simulator. Furthermore, based on the RC tree's structural and physical properties, we devise tree contrastive (TContrast) loss to guide the feature transformation during graph learning. The results show that our model reaches state-of-the- performance compared

---

6.18
Compuraton Timelo
MAE Change with Transier Learning
Layers
Delay
Slew
Slew to Delay
Synthetic
RISC-V
Synthetic
RISC-V
6.00
0.9174
0,0099
0.0007
1.5981
0.00214
0.3267
0,00121
16
2.3044
0.00321
0.3437
0.00261
1.876]
0.00541
0.9697
0.00181
64
2,0681
0.00371
0.5051
0.00341
Table 4: SyncTREEE Transfer Experiments Between De-
8
10
12
14
16
18
Cirout
lay Prediction and Slew Prediction. (L means "lower better")
Figure 7: Runtime of SPICE simu- lation and SyncTREE model with circuit size.
with prospective GNNs and outperforms the SPICE simulator in computation efficiency. Therefore, our SyncTREE model can better support fast incremental timing updates. Compared to SPICE's full-fledged simulation even for incremental circuit updates, our solution's incremental updates will be even more appealing to IC design optimization.
Acknowledgment
This work was supported, in part, by the SUNY-IBM AT Collaborative Research Alliance Project #22003, the SUNY Empire Innovation Program, and, by Advantest as part of the Graduate School "Intelligent Methods for Test and Reliability" (GS-IMTR) at the University of Stuttgart.
References
[1] Guyue Huang, Jingbo Hu, Yifan He, Jialong Liu, Mingyuan Ma, Zhaoyang Shen, Juejian Wu, Yuanfan Xu, Hengrui Zhang, Kai Zhong, et al. Machine learning for electronic design automa- tion: A survey. ACM Transactions Design Automation of Electronic Systems (TODAES), 26(5):1 16, 2021.
[2] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Yu Philip. comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems. 32(1):4 24, 2020.
[3] Haoxing Ren, Siddhartha Nath, Yanqing Zhang, Hao Chen, and Mingjie Liu. Why are graph neural networks effective for eda problems? In Proceedings of the 41st IEEE/ACM International Conference Codduter-Aided Design, pages 1-8, 2022.
[4] Zizheng Guo, Mingjie Liu, Jiaqi Gu, Shuhan Zhang, David Z Pan, and Yibo Lin. A timing engine inspired graph neural network model for pre-routing slack prediction. In Proceedings of the 59th ACM/IEEE Design Automation Conference, pages 1207-1212, 2022.
[5] Mingjie Liu, Walker Turner, George Kokai, Brucek Khailany, David Pan, and Haoxing Ren. Parasitic-aware analog circuit sizing with graph neural networks and bayesian optimization. In 2021 Design, Automation Test Europe Conference & Exhibition (DATE), pages 1372-1377. IEEE, 2021.
[6] Daniela Sánchez Lopera, Lorenzo Servadei, Gamze Naz Kiprit, Souvik Hazra, Robert Wille, and Wolfgang Ecker. A survey of graph neural networks for electronic design automation. In 2021 ACM/IEEE 3rd Workshop on Machine Learning CAD (MLCAD), pages 1-6. IEEE, 2021.
[7] Yifan Sun, Changmao Cheng, Yuhan Zhang, Yichen Wei. Cirele loss: A unified perspective of pair similarity optimization. Proceedings of the JEEE/CVF conference computer vision and pattern recognition, pages 2020.
Chi Zhang, Liang Zheng, Zhongdao Wang, and
10

---

[8] Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. Deepgens: Can gens go deep as cnns? In Proceedings of the IEEE/CVE international conference on computer vision, pages 9267-9276, 2019.
[9] Zhanghao Wu, Paras Jain, Matthew Wright, Azalia Mirhoseini, Joseph E Gonzalez, and Ion Stoica. Representing long-range context for graph neural networks with global attention. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[10] Rajat Talak, Siyi Hu, Lisa Peng, and Luca Carlone. Neural trees for learning on graphs. Advances in Neural Information Processing Systems, 34:26395-26408,, 2021.
[11] Yi-Chen Lu and Sung Kyu Lim. On advancing physical design using graph neural networks. In Proceedings of the 41 IEEF/ACM International Conference on Computer-Aided Design, pages 1-7, 2022.
[12] Haoxing Ren, George F Kokai, Walker Turnne, and Ting-Sheng Ku. Paragraph: Layout parasitics and device parameter prediction using graph neural networks. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1-6. IEEE, 2020.
[13] Yaguang Li, Yishuang Lin, Meghna Madhusudan, Arvind Sharma, Wenbin Xu, Sachin Sapatnekar, Ramesh Harjami, and Jiang Hu. A customized graph neural network model for guiding analog ic placement. In Proceedings of the 39th International Conference on Computer- Aided Design, pages 1-9, 2020.
[14] Kourosh Hakhamaneshi, Marcel Nassar, Mariano Phielipp. Pieter Abbeel, and Vladimir Sto- janovic. Pretraining graph neural networks for few-shot analog circuit modeling and design. IEEE Transactions on Computer-Aider Design of Integrated Circuits and Systems, 2022.
[15] Zhiyao Xie, Rongjian Liang, Xiaoqing Xu, Jiang Hu, Yixiao Duan, and Yiran Chen. Net2: A graph attention network method customized for pre- pre-placement net length estimation. In Proceedings of the 26th Asia and South Pacific Design Automation Conference, pages 671-677, 2021.
[16] Kevin Kai-Chun Chang, Chun-Yar Chiang, Pei- Yu Lee, and iris Hui-Ru Jiang. Timing macro modeling with graph neural networks. In Proceedings of the 59th ACM/IEEE Design Automation Conference, pages 1219-1224, 2022.
[17] Robert Kirby, Saad Godil, Rajarshi Roy. and Bryan Catanzaro. Congestionnet: Routing congestion prediction using deep graph neural networks. In 2019 IFIP/IEEE 27th International Conference on Very Large Scale Integration (VLSI-SoC), pages 217-222. IEEE, 2019.
[18) Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Wenjie Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Azade Nazi, et al. A graph placement methodology for fast chip design. Nature, 594(7862):207-221 2021.
[19] Yi-Chen Lu, Sai Surya Kiran Pentapati, Lingjun Zhu, Kambiz Samadi, and Sung Kyu Lim. Tp-gnn: A graph neural network framework for tier partitioning in monolithic 3d ics. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1-6. IEEE, 2020.
[20] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in neural information processing systems, 14:5812-5822 2020.
[21] Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. Adversarial graph augmentation to improve graph contrastive learning. Advances Neural Information Processing Systems, 34:15920- 15933, 2021.
[22] Zekun Tong, Yuxuan Liang, Henghui Ding, Yongxing Dai, Xinke Li, and Changhu Wang. Directed graph contrastive learning. Advances Neural Information Processing Systems, 14:19580-19599
2021.
[23] Yuyang Wang. Jianren Wang, Zhonglin Cao, and Amir Barati Farimani. Molecular contrastive learning of representations graph neural networks. Nature Machine Intelligence, 4(3):279- 287, 2022.
11

---

[24] Yiyue Qian, Chunhui Zhang, Yiming Zhang, Qianlong Wen, Yanfang Ye, and Chuxu Zhang. Co-modality graph contrastive learning for imbalanced node classification. Advances Neural Information Processing Systems, 13:1582--5874, 2022.
[25] Laurence W. Nagel and D.O. Pederson. Spice (simulation program with integrated circuit emphasis). Technical Report UCB/ERL M382, EECS Department, University Califomia, Berkeley, Apr 1973.
[26] Chung-Wer Wen Ho, Albert Ruehli, and Pierce Brennan. The modified nodal approach to network analysis. IEEE Transactions on circuits and systems, 22(6):504-509, 1975.
[27] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10003. 2017.
[28] Jorge Rubinstein, Paul Penfield, and Mark Horowitz. Signal delay in re tree networks. IEEE Trunsactions on Computer-Aided Design of Integrated Circuits and Systems, 2(3):202-211,
1983.
[29] Thomas N Kipf and Max Welling. Supi-supe classification with graph convolutional networks. arXiv preprint arXiv:1609.0283 2016.
[30] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
[31] Lloyd N Trefethen and JAC Weideman. The exponentially convergent trapezoidal rule. siam REVIEW, 56(3):385 458, 2014.
12

---

A Appendix
A.1 Message Passing in SyncTREE
As the message passing process in Figure information from leaves, sub-branches, and the whole global structure is first collected following bottom-up propagation by GAT Then, the final node representations of GA are applied to each layer of GAT to jointly update the node attributes of the corresponding top-down tree with the node embeddings of its previous layer. As the example shown in Figure [ by designing this two-pass meessage-psssing passing mechanism, the node features will incorporate the information from different levels and become more expressive. Furthermore, in the top-down tree, the root node only be updated with synchronized hy since it doesn't 't have any incoming connection, it ensures that information injection at the source of the top-down tree is fixed which can help to maintain differentiable feature embeddings without over-smoothing.
Leaf
Leaf
Sub-branch
Suub branch
Gliau!
syne
up
O
node
top-down
6
hiu h-h
top-down
Figure 8: Illustration of our two-pass message-passing mechanism.
A.2 Synthetic and RISC-V Dataset Preparation
Our dataset is composed of artificially generated and practical RC trees and the golden timing results sinks (leaf nodes of each RC tree) obtained by SPICE simulation. On the one hand, we follow the pipeline in Figure to generate the synthetic dataset. To be specific, we first adopt Algorithm to generate RC-trees with random typologies and then convert them to artificial IC interconnects for further SPICE timing measurement. On the other hand, we directly extract RC trees from practical RISC-V circuit designs compose the RISC-V V dataset.
Generate (randomly set RC-valucs and structural typology)
Build corresponding notlist for RC tree
different driving voltage
Invoke HSPICE
Parse SPICE results
Figure 9: Pipeline for the synthetic dataset generation.
A.3 Baselines's Implementation
The GNNs of all the baseline models are set with 32/128 hidden dimensions separately for the synthetic dataset/RISC-V dataset. For GraphTrans, the dimension of the feedforward full-connection layers in the Transformer of GraphTrans is set to 256 with 0.1 dropout probability between layers, the number of attention heads is set to and the max input sequence length is set to the maximum eircuit size. It should be noted that we only made little modification to the Graph Trans model. GraphTrans
13

---

Algorithm 1 Generate artificial RC-trees
Require:
E [Umin, Umon]. R E [Rmins Rmaal, , C' E [Cminy Caaz]
Initialize voltage Ga driving cell, edge type (rising or falling), depth D of RC tree parent set = list [driiinniee
parent &- randomly pick one element from porent set.
while depth D do
randomly choose R,
generate child, add child into parent set
the Rebstd of the edge from child to the parent t- R.
the Cehild of child to the ground C
parent &- randomly pick one element from parent set
D +
end while
is originally designed for node classification tasks, it takes CLS token from Transformer output as the representation of the whole graph and applies a linear module followed by softmax to implement prediction. In order to incorporate global information into node features, in our experiments, we concatenate the CLS token with node embeddings and then feed into MLP get the final output. For NTREE, we set GAT as its basic block with a 0.2 dropout probability between layers. We follow the original junction-tred traeb---eee- algorithm in [10 to compose H-trees from our RC circuits with the same radius setting for extracting subgraphs in the paper.
A.4 Analysis of TContrast Loss To visualize the converging process during training, we plot the distribution of similarity pairs in space at different epochs in Figure IO It obviously shows that our model approaches the optimization goal with more concentrated similarity distribution after enough training with the guidance of TContrast loss. In Figure [ we show the MAE difference of timing results obtained by vanilla TREE and TC-loss guided SREE As shown in the results, after being combined with TC loss, our SyncTREE model has smaller errors for most types of RC trees which can effectively prove the validity of TC loss.
1.0
10
1.00
08
0.75
0.6
0.30
02
0.5
02
00
0.00
0.25
0.50
0.0
(a) Epochel
(b) Epach=20
(c) Eguch=40
(d) Epoch=60
Figure 10: The distribution of similarity pairs with training epochs.
(a) Delay on S
(b) Slew on
(c) Delay on
(d) Slew on R
Figure 11: Mean Average Error difference after applying TContrast loss on the synthetic dataset (S) and RISC-V V dataset (R). (Negative values indicate that los---ddddd SyncTREE has a lower MAE error than vanilla SynCTREE)
14


Reviews:
[{"review_id": "VB5Z1qSZUr", "reviewer": "Reviewer_ZUNz", "summary": "The paper proposes a GNN model, dubbed SyncTREE, for IC's RC-tree timing analysis. Two techniques are proposed: 1) two-pass message-passing and 2) Tree Contrastive loss. Experiments of two IC designs demonstrate the best accuracy of SyncTREE over other SOTA GNN models. ", "questions": "1. I am confused by the middle part of Fig. 2 w/ \"Init Nodes\". The hidden features from the bottom-up graph are used to initialize those in the top-down graph. Why there is an arrow from the top-down graph to the \"Init Nodes\"? In addition, the right-most part of Fig. 2 shows a \"mask\", which is given w/o explanation.  \n\n2. What is the rule of thumb for setting the number of hidden dimensions? In addition, it would be better if the authors can discuss the selection of the number of layers. These hyperparameters can influence the accuracy and speed trade-offs and a discussion can potentially ease the use of the proposed SyncTREE. \n\n3. Fig.7 provides the computation time vs. circuit size. What are the hardware platforms for SPICE simulation and SyncTREE model? Since given a new circuit, new graph training is needed. It would be better if the author can also provide the training time vs. circuit size for better show the efficiency of the proposed SyncTREE. \n\n4. Typos. E.g., \"news ways\" should be \"new ways\" in the Abstract section. Line # 216, \"We\" should be \"we\". \n\n5. Considering that timing analysis is an important problem for EDA community and considering existing efforts to use GNN for EDA tasks, can conferences such as DAC and ICCAD be a more suitable platform for publishing this paper? ", "limitations": "The authors do not discuss the limitations of the proposed method. It would be better if the authors can provide the limitations and the potential future directions of this work. ", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Domain-specific knowledge is used in SyncTREE's two techniques. I think these two techniques are original.\n\n2. Evaluations demonstrate the performance of SyncTREE's two techniques. The timing analysis shows the promising potential speedup of SyncTREE over traditional SPICE with increased circuit sizes. ", "weaknesses": "1. There is no clear guideline on setting the number of hidden dimensions and layer selection, which can influence accuracy and speed trade-offs for the practical adoption of the proposed SyncGNN. \n\n2. I'm pondering if an EDA-focused conference like DAC or ICCAD may be a more suitable platform for this paper. Given that the development of SyncGNN draws heavily on EDA-related domain-specific knowledge, such a conference could provide a more targeted audience and potentially foster more fruitful discussions for further refining this approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer #ZUNz \n\nWe greatly appreciate your careful and detailed review. Here are some points we would like to clarify:\n\n### Q1. Explanation of “Init Nodes” & “mask” in Fig. 2.\n\nThe \"Init Nodes\" part of Fig. 2 involves the node attributes assignment of the top-down graph after each convolutional layer. In our two-pass message-passing mechanism, as shown in Equation (5), the node attributes in the top-down graph at $l+1$ layer will be updated by two parts: 1) the node representations obtained by the $AGG^{l}_{td}$; 2) the final node embeddings of the bottom-up graph. With this update mechanism, the node attributes will be reassigned accordingly for the next layer.\n\nThe mask in Fig. 2 is used to filter out leaf nodes for calculating the loss function and obtaining the final output. Since our task is to predict propagation delay/slew of RC trees, we take the final representations at leaf nodes (sinks) as the prediction results. That's why we apply masks to improve the model performance by reducing the influence of irrelevant information in the loss function.\n\n### Q2. Rules regarding setting hidden dimensions and the number of convolution layers.\nThe hyperparameter selection of hidden dimensions and the number of convolution layers directly determine the model performance. A lower hidden dimension might lead to underfitting, leading to the failure of the model to capture complex patterns in the graph data. Conversely, a higher hidden dimension could lead to overfitting. Besides, the number of convolutional layers in GNNs determines the depth of information propagation across the graph. To choose an ideal number of convolution layers, the depth of input graphs is a crucial factor to consider. In our work, the choice of hidden dimensions and the number of convolution layers is decided by the trade-off between model accuracy and efficiency. By systematically exploring different parameter combinations and analyzing the model’s behavior, we then make an informed decision about the appropriate number of layers and dimensions to apply.\n\nFollowing the reviewer’s suggestion, we evaluate the inference time and model accuracy on the Synthetic and RISC-V datasets under different parameter combinations. The results are shown in the following tables (Note: 4L refers to 4 convolution layers, 32D refers to 32 hidden dimensions).\n \nTable 1 MAE ps (Inference Time s) on Synthetic Dataset (batches: 31, batch size: 32)\n|        |  4L         |  8L         | 16L         | 32L         | 64L         |\n|--------|------------|------------|-------------|-------------|-------------|\n|  32D   | 8.745 (0.010) | 6.631 (0.018) | 3.775 (0.035) | **3.424** (0.068) | 3.556 (0.141) |\n|  64D   | 9.075 (0.014) | 7.258 (0.026) | 4.886 (0.051) | 4.662 (0.102) | 4.563 (0.204) |\n| 128D   | 9.564 (0.022) | 6.986 (0.041) | 4.737 (0.079) | 4.419 (0.157) | 5.106 (0.314) |\n\n\nTable 2 MAE ps (Inference Time s) on RISC-V Dataset (batches: 647, batch size: 128)\n|       | 4L              | 8L              | 16L             | 32L               | 64L              |\n|-------|-----------------|-----------------|-----------------|-------------------|------------------|\n| 32D   | 0.0385 (0.017)  | 0.0569 (0.021)  | 0.0213 (0.039)  | 0.0145 (0.075) | 0.0244 (0.136)  |\n| 64D   | 0.0352 (0.021)  | 0.0271 (0.034)  | 0.0149 (0.056)  | 0.0120 (0.103) | 0.0236 (0.202)  |\n| 128D  | 0.0313 (0.033)  | 0.0195 (0.054)  | 0.0128 (0.096)  | **0.0106** (0.160) | 0.0176 (0.312)  |\n\nGiven the above statistics, we then separately set 32 hidden dimensions and 128 hidden dimensions for our model over Synthetic and RISC-V dataset.\n\n### Q3. 1) Hardware platforms running SPICE & our experiments; 2) Plotting training time vs. circuit size. \n1) All experiments in this paper are implemented with PyTorch 1.13.1 and PyTorch Geometric 2.2.0 frameworks, and executed on a Ubuntu server equipped with Intel Xeon Gold 6330 CPU with 56 cores/2 threads running at 2.0GHz. The reference SPICE simulations are carried out with the commercial Synopsys HSPICE simulator on an AMD Ryzen 3950X with 16 cores/32 threads at 3.5GHz.\n\n2) As shown in Table 1, Fig. 10, and Fig. 11, our training and validation dataset is composed of RC trees with different sizes and typologies. During model training, all circuit samples are shuffled randomly to reduce bias and improve generalization. Technically, a training batch is composed of various circuits of different sizes and typologies, so it's hard to get the exact training time regarding specific circuit sizes. Moreover, since graph neural networks can deal with unstructured data, if given a new circuit, we don't need to retrain the model. Actually, in the inference stage of our model, the input circuit graphs are new and not seen during training.\n\n### Q4. Typos correction.\nWe appreciate your keen attention to pointing out the typos and apologize for any oversight on our part. We will correct the errors you mentioned in the revision.\n\n### Q5. A more suitable platform for this paper.\n\nWe appreciate your thoughtful consideration of the paper's relevance to different conference platforms within the EDA community. However, we believe that NeurIPS remains a suitable platform for the publication of our work for the following reason. \n\nOur work is mainly about physics-informed deep learning, which combines principles from physics in a specific field with deep learning techniques, and aligns well with the interdisciplinary nature of NeurIPS. To give examples, there are some recent EDA-related papers accepted by premier AI conferences, which makes us firmly believe NeurIPS can effectively draw attention from both the machine learning and EDA communities and be more confident to publish our work through this platform.\n\n\n"}, {"review_id": "rpIsSUdult", "reviewer": "Reviewer_6pFT", "summary": "This paper proposes a GNN based method that specializes in timing analysis.", "questions": "* For different circuit technology sizes, does the proposed method need to be trained differently? or is the technology sizing part of the input?\n", "limitations": "* There is no explicit discussion on limitations.", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* This paper is well written and organized, easy for readers to follow.\n* Related background, related work that uses GNN on circuits are discussed with sufficient level of detail.\n* The core problem looks well formatted.\n* Experimental results looks promising with benchmarks on RISC-V, offering faster than SPICE simulation looks appealing.\n* Performance and results are thoroughly analyzed.\n", "weaknesses": "Please see questions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer #6pFT\nThank you for your feedback and support! We hope these responses satisfactorily answer your questions:\n### Q1. Model adaptation to different circuit size.\nIn this paper, our motivation is to devise a timing prediction model that can be applied to different circuits by leveraging the representation capability of Graph Neural Networks for unstructured data. Our model is trained on the dataset containing various RC circuits of different sizes and typologies, which means that our model doesn’t need to be trained differently regarding different circuit sizes.\n\n"}, {"review_id": "ENIbmUwAFm", "reviewer": "Reviewer_sZvh", "summary": "This manuscript present a SyncTree to speed up timing analysis in IC design. ", "questions": "Check Weakness section", "limitations": "Check Weakness section", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "-The problem that the manuscript is trying to address is important (increase speed of timing analysis)\n-Evaluation and comparison to other machine learning based methods. \n", "weaknesses": "- It is unclear are the Mean Average Error with respect to Spice simulation?\n- Table 2 and Table 3 should consist of time it takes to perform the predictions. \n- Additionally, what is the increase in speed between SyncTree and spice simulation. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer #sZvh\nThank you for your comments and the time spent reviewing the work. We try to address all the raised points in the following content. \n### Q1. Clarification about MAE.\nIn our experiments, we treat the SPICE simulation measurements as the golden timing results. The Mean Average Error (MAE) is measured between predictions and spice simulation results.\n### Q2. Running time of results in Table 2 and Table 3.\nWe additionally record the inference time (s) of results in Table 2 and Table 3 as follows:\n1. Table 2\n|    | GCN    | GAT    | GraphSAGE | DeepGCN | GraphTrans | NTREE | SyncTREE |\n|----|--------|--------|-----------|---------|------------|-------|----------|\n| 4L | 0.003  | 0.005  | 0.002     | 0.008   | 0.060      | 0.024 | 0.015    |\n| 8L | 0.005  | 0.009  | 0.004     | 0.015   | 0.055      | 0.038 | 0.025    |\n| 16L| 0.011  | 0.020  | 0.007     | 0.029   | 0.059      | 0.072 | 0.044    |\n| 32L| 0.019  | 0.039  | 0.013     | 0.055   | 0.062      | 0.114 | 0.100    |\n| 64L| 0.036  | 0.071  | 0.025     | 0.112   | 0.056      | 0.233 | 0.185    |\n\n2. Table 3\n|     | GCN    | GAT    | GraphSAGE | DeepGCN | GraphTrans | NTREE | SyncTREE |\n| --- | ------ | ------ | --------- | ------- | ---------- | ----- | -------- |\n| 4L  | 0.007  | 0.008  | 0.006     | 0.018   | 0.101      | 0.057 | 0.035    |\n| 8L  | 0.010  | 0.015  | 0.008     | 0.033   | 0.099      | 0.110 | 0.051    |\n| 16L | 0.017  | 0.031  | 0.015     | 0.070   | 0.085      | 0.193 | 0.097    |\n| 32L | 0.033  | 0.060  | 0.029     | 0.131   | 0.084      | 0.245 | 0.162    |\n| 64L | 0.0659 | 0.1172 | 0.0665    | 0.269   | 0.087      | 0.317 | 0.294    |\n\n### Q3.  Improvement in speed of SyncTREE.\nAs shown in Fig. 7, we evaluate the computation efficiency of SyncTree model and SPICE simulation along with the circuit size. It shows that SyncTREE is significantly faster than SPICE with this advantage increasing as the circuit size grows. We further analyze the computation complexity towards SPICE and SyncTREE in the Computation Efficiency Section of Results and Discussion Part. To conclude, SPICE needs to solve DAEs in a time-incremental manner to simulate the circuit behavior, which involves intensive matrix operations like LU decomposition. In contrast, SyncTREE leverages Graph Attention Networks which only revolve around linear transformation, leading to a much faster running speed.\n\n"}, {"review_id": "6hAHO23D5I", "reviewer": "Reviewer_YDvu", "summary": "This papers proposes SyncTREE that uses a bottom-up and top-down graph attention network with a Tree Contrastive Loss to predict the delay and slew for IC interconnects. Compared to other GNN methods, the proposed one achieve lower prediction error across synthetic and RISC-V benchmarks.", "questions": "Major questions are summarized above. Basically, more ablation studies are required. More discussion on generalization to different circuits and data efficiency is expected. I will consider increasing the score if the major concerns above are addressed.", "limitations": "No.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe proposed method is quite novel and leverages the prior knowledge in timing analysis in network design and loss function design, achieving the best results on both synthetic and RISV benchmarks.", "weaknesses": "1.\tThe modified aggregation mechanism claims to preserve resistance information by linearly combining node and edge features. However, no ablation study or evidence on this technique has been shown. Similarly, the network designs, e.g., residual connections from bottom-up tree, two directed graphs, or other introduced techniques are not evaluated through ablation studies. It seems that the benefits are mainly from the directed graphs, not GAT. How does this method perform if applied to other GNNs listed in Table 3?\n2.\tThe layers in the trained GAT is fixed, not adaptive to different circuits. If a RC tree is deeper than the GAT network, then the timing information cannot be propagated from source to sink. The generalization raises some concerns in this case.\n3.\tIn line 266, it claims the proposed method achieves better accuracy on larger circuits. However, Figure 4(d) seems the relative prediction error get worse with larger circuit sizes if I understand correctly. Why the performance gets worse as circuit size changes is not explained. How to solve this generalization gap should be a key question to investigate for this work.\n4.\tThe runtime comparison need to clarify the hardware platform. If the GAT uses GPU, then it might be a unfair comparison if compared to CPU-version SPICE. There exists GPU version LU factorization to accelerate matrix inversion.\n5.\tThe prediction error lacks intuitive analysis. Whether 0.05 ps MAE considered to be large or small enough for timing analysis is not clear. In other words, the significance of the results is unclear. How does this delay and slew error ultimately affect critical path delay or total negative slack should be discussed. The variance in the predicted error on each node is also important compared to MAE.\n6.\tIn terms of generalization and transfer learning, transferring to different benchmark/circuit are expected instead of transfer from delay to slew prediction. How does the physics-informed loss function helps to increase data efficiency compared to pure data-drive method is not shown. The generalization and data efficiency are critical concerns for ML-based PDE solving tasks, which are not deeply explored in this paper.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer #YDvu\nThank you for carefully going through the paper and pointing out so valuable questions. We hope these responses satisfactorily answer your questions:\n\n### Q1. Ablation study.\n1. Ablation study toward GAT modification.\n\nFollowing the reviewer’s suggestion, we evaluate our model with/without modification towards the GAT aggregation mechanism (GAT-Mod). The results are as follows:\n\nTable 1 MAE ps on Synthetic Dataset w/wo GAT-Mod\n|             | 4 Layers | 8 Layers | 16 Layers | 32 Layers | 64 Layers |\n|-------------|---------:|---------:|----------:|----------:|----------:|\n| W-GAT-Mod   |   **8.745**   |   **6.631**   |    **3.775**    |    **3.424**    |    **3.556**    |\n| WO-GAT-Mod  |    9.752    |    6.589    |     4.886     |     4.597     |     4.507     |\n\nTable 2 MAE ps on RISC-V Dataset w/wo GAT-Mod\n  |            | 4 Layers | 8 Layers | 16 Layers | 32 Layers | 64 Layers |\n  |------------|----------|----------|-----------|-----------|-----------|\n  | W-GAT-Mod  | **0.0313** | **0.0195** | **0.0128** | **0.0106** | **0.0176** |\n  | WO-GAT-Mod | 0.0395   | 0.0325   | 0.0274    | 0.0271    | 0.0294    |\n\n2. Other discussions.\n- For the residual connections from the bottom-up tree, if we remove this part, the model will degrade to conventional GAT working on a directed graph. In this case, the leaf nodes only can gather information from the propagation path, leading to the loss of high-level substructure features.  \n- About applying two directed graphs to other GNNs, let us take the results in Table 2 and 3 as reference. Among all baseline models, GAT presents higher accuracy. That's why we use GAT as the basic block in our SyncTREE model. \n\n### Q2. Concern about the depth of RC trees and the number of convolution layers.\nIt’s true that if an RC tree is deeper than the GAT network, then the timing information cannot be propagated from source to sink. This is a typical issue of message-passing GNNs caused by its inner mechanism. A deeper graph structure might necessitate a deeper GNN architecture to capture long-range dependencies. To show the benefit of adding layers to deeper RC trees and the accuracy changes it brings to shallower circuits, we analyze the relative error regarding RC tree depth and size under different model depths. We attach this result in the pdf of the global response, please refer to Fig.1 and Fig. 2. \n\n### Q3.1) Clarification about Fig 4d’s description; 2) Generalization problem. \n1) We are sorry for our unclear description regarding Fig. 4d’s description. The results in Fig. 3 and Fig. 4 show that our approach demonstrates enhanced accuracy when applied to circuits of greater dimensions and more sinks. \n\n2) To evaluate our model's performance thoroughly, we make two highly diverse datasets, as shown in statistics shown in Fig. 10 and Fig. 11 in our supplementary material. Our model is trained on datasets containing various RC circuits of different sizes and typologies. Compared with golden delay, the value range of golden slews is more limited, which makes the model tend to predict the small slew values very well as shown in Fig. 4d. However, delays are highly varied across circuit sizes, for example, delays of tiny circuits (eg. RC trees that have less than 4 nodes) are only at 1e-4 ps level, leading to higher errors for these circuits as shown in Fig. 3d. It should be pointed out that these tiny circuits are not common in normal IC designs but deserve to be investigated with our model. As for the results in Fig. 4d showing increasing error with larger circuit sizes, we believe the overall accuracy is still quite satisfactory considering the high diversity of our dataset.\n\n## Q4. Running Platform Details.\nWe report this question in the global response. \n\n## Q5. 1) Interpretation towards MAE value; 2) Significance of our work; 3) Error impact on Critical path delay.\nWe report problems 1 and 2  in the global response. \n\n3) We follow the reviewer’s suggestion and calculate the critical delay MAE on both two datasets. The critical delay MAE is the average of absolute error between the golden critical delays and the predictions at the critical sinks across all circuits in the dataset. Please note the average critical delays on Synthetic dataset and RISC-V dataset are 183.25 ps and 0.68 ps respectively.\n\n|                         |  4L   |  8L   | 16L   | 32L   | 64L   |\n|-------------------------|-------|-------|-------|-------|-------|\n| Synthetic Critical-MAE  | 78.828| 59.976| 32.004| **24.837** | 28.661|\n| RISC Critical-MAE       | 0.4034| 0.2170| 0.1311| **0.1214** | 0.1283|\n\nFrom the result, we can observe that the critical delay MAE is closely related to overall MAE. \n\n## Q6. 1) Transfer learning to different benchmarks/circuits; 2) The advantage over data efficiency of Physics-informed loss function.\n1) Our model is trained on datasets containing various RC circuits of different sizes and typologies, which means that our model doesn’t need to be trained differently regarding different circuit sizes. Besides, our two benchmark datasets are from different sources, thus we have different hidden dimension settings to optimize the accuracy correspondingly, so it's not feasible do transfer learning to different benchmarks due to the model difference and dataset inconsistency. \n\n2)  In terms of data efficiency, we implement an additional experiment to evaluate the possible benefit of TC loss under different training set percentages. \n\n|        | Synthetic-W-TC | Synthetic-WO-TC | RISC-V-W-TC | RISC-V-WO-TC |\n|--------|---------------|----------------|-------------|--------------|\n| 25%    | 4.2838        | 4.5682         | 0.0311      | 0.0342       |\n| 50%    | 4.0534        | 4.0246         | 0.0166      | 0.0228       |\n| 75%    | 3.9063        | 3.8453         | 0.0149      | 0.1912      |\n\nFrom the result, we can observe that SyncTREE with TC loss exhibits obvious benefits over data efficiently on the RISC-V dataset. \n\n\n\n\n\n\n"}]

Author rebuttal:
## General Response for Common Questions
Thanks for all reviewers' constructive suggestions, which help us find some points we didn’t explain clearly. We believe it’s necessary to make some global clarifications toward the following points:

### 1. Significance of our work.
Timing analysis is crucial for ensuring the proper functioning, performance, and reliability of integrated circuits. Fast and accurate timing prediction can greatly reduce the runtime overhead, which is very significant for IC design. In this paper, we propose a tree-based graph neural network, SyncTREE, to speed up the timing analysis, the significance of our work is listed below:

- To the best of our knowledge, this is the first work applying GNNs to directly make timing predictions, which can offer a good baseline for future works.
- Our model extends beyond conventional GNNs and presents the best representation power for RC trees. 
- Compared with SPICE, our model achieves satisfactory accuracy with far less runtime overhead, which means our model has the potential to expedite the IC design process. 
- Conventional timing analysis tools like static timing analysis (STA) have limited parallelism capability, slowing down analysis and optimization tasks, while our model is GAT-based and thus can be parallelized easily.

### 2. Dataset clarification.
Our Synthetic dataset and RISC-V dataset are composed of various circuits having different sizes and structures. In our works, we devise a timing prediction model that can be applied to different circuits by leveraging the representation capability of Graph Neural Networks for unstructured data. When given a new and unseen circuit, we don’t need to retain the model.

### 3. Running platform information
All experiments in this paper are implemented with PyTorch 1.13.1 and PyTorch Geometric 2.2.0 frameworks and executed on a Ubuntu server equipped with Intel Xeon Gold 6330 CPU with 56 cores/2 threads running at 2.0GHz. The reference SPICE simulations are carried out with the commercial Synopsys HSPICE simulator on an AMD Ryzen 3950X with 16 cores/32 threads at 3.5GHz.

### 4. MAE metric interpretation
Acceptable propagation delay MAE depends on the specific requirements and performance criteria of the circuit and its intended application. Some applications, such as high-performance computing or communication systems, demand minimizing propagation delay error to ensure accurate timing and reliable operation. Considering the current highest clock rate of CPU doesn’t surpass 10 GHz, which means the time unit for one execution cycle of CPU is above 100 ps. Therefore the 0.05 ps error example reviewer #YDvu posted for IC design is pretty small. Moreover, it should be pointed out that since our goal is to compare with SPICE from both accuracy and computation cost, we set timing results obtained by SPICE simulation as golden and present the percent error between SPICE and our model in Fig. 3 (b) (d) and Fig.4 (b) (d).





Meta-review: Summary:

This paper proposes SyncTREE which uses a bottom-up and top-down graph attention network with a Tree Contrastive Loss to predict the delay and slew for IC interconnects. Compared to other GNN methods, the proposed one achieves lower prediction error across synthetic and RISC-V benchmarks.

Strengths:

1 - Domain-specific knowledge is used in SyncTREE's two techniques which are original.

2 - Evaluations demonstrate the performance of SyncTREE's two techniques. The timing analysis shows the promising potential speedup of SyncTREE over traditional SPICE with increased circuit sizes.

3 - This paper is well-written and organized, easy for readers to follow.

4 - Related background, related work that uses GNN on circuits are discussed with sufficient level of detail.

5 - The core problem looks well formatted.

6 - Experimental results look promising with benchmarks on RISC-V, offering faster than SPICE simulation looks appealing.

7 - Performance and results are thoroughly analyzed.

8 - The problem that the manuscript is trying to address is important (increase speed of timing analysis)

9 - The proposed method is quite novel and leverages the prior knowledge in timing analysis in network design and loss function design, achieving the best results on both synthetic and RISV benchmarks.

Weaknesses:

1 - There is no clear guideline on setting the number of hidden dimensions and layer selection, which can influence accuracy and speed trade-offs for the practical adoption of the proposed SyncGNN.

2 - No ablation study or evidence on this technique has been shown.

3 - The layers in the trained GAT is fixed, not adaptive to different circuits. If a RC tree is deeper than the GAT network, then the timing information cannot be propagated from source to sink. The generalization raises some concerns in this case.

4 - The runtime comparison needs to clarify the hardware platform.

5 - The prediction error lacks intuitive analysis.

Decision:

A majority of reviewers vote for acceptance. The only review favouring rejection is very short and of low quality and the reviewer did not respond to the authors' rebutall or comments by the area chair. This reviewer also does not give a clear reason for rejection. I, therefore, decided to ignore that negative review. Based on this, I decide to accept the paper and encourage the authors to use the feedback provided to improve the paper for its camera ready version.
Decision: Accept (poster)