SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 1: Statistics of Datasets
Synthetic Dataset	RISC-V Dataset
Configuration	uniform distribution	min	average	max
Nodes	2-51 nodes	2 nodes	6.21 nodes	20 nodes
Resistance	10-2000 ohm	0.5 ohm	101.6 ohm	549.1 ohm
Capacitance	0.01-2 fF	0.00113 fF	0.192 fF	2.398 fF
Samples	4,066 circuits	414,639 circuits
Synthesized Dataset. The dataset consists of synthesized RC circuits with various circuit typologies, including rare interconnections. To generate these circuits, we first generate an RC tree and then simulate its corresponding artificial IC interconnects with SPICE to obtain the golden timing values. The pseudocode for generating RC trees, the workflow for data preparation, and the golden timing distribution in data samples are provided in the Appendix.
RISC-V dataset. The dataset is composed of real RC circuits extracted from practical RISC-V IC designs. Our objective is to validate the effectiveness of our model in analyzing the timing of interconnects in practical IC designs.
Models and parameters. For our SyncTREE model, we separately set 32 hidden dimensions on the synthetic dataset and 128 hidden dimensions on the RISC-V dataset. The TContrast loss is set with hops of 2,  $N$  of 1,  $m$  in the range of [0.1, 0.5], and  $B$  is 64 for the synthetic dataset and 128 for the RISC-V dataset. All the models are trained with the Adam optimizer  $(\beta_{1} = 0.9, \beta_{2} = 0.99)$ . The batch size is set to 32 for the synthetic dataset and 256 for RISC-V dataset. The learning rate was set to 8e-4 for 4/8/16-layer models and 4e-4/2e-4 for 32/64-layer models. All the models are trained for 10 epochs on the RISC-V dataset and 60 epochs on the synthetic dataset.
Baselines. Considering timing analysis is a unique task that depends on the structures of RC trees at different levels, we compare our method to various conventional GNNs, including GCN [29], GAT [27], and GraphSAGE [30], as well as prospective GNNs such as DeepGCN[8], GraphTrans[9], and NeuralTree[10]. For GraphTrans, to incorporate complex global information into node features, we concatenate the CLS token embedding of Transformer with node embeddings and input to MLP to get the final output.
In this section, we present a comprehensive evaluation of our SyncTree model's performance compared to selected GNN baselines for delay and slew prediction tasks and showcase the impact of using TContrast loss function for training. Then, we discuss the computational efficiency of SyncTREE and its transfer learning capabilities.
Main Results. We present the delay and slew prediction results of different GNN models on the Synthetic and RISC-V datasets in Table 2 and Table 3. The results show that SyncTREE outperforms the existing GNN baselines across all model depths on both synthetic and RISC-V datasets. For example, even the only four-layer SyncTREE model nearly surpasses all the other baselines with different model depths. Furthermore, our model consistently outperforms the baselines on the RISC-V dataset, highlighting its applicability to real-world scenarios. To evaluate the effectiveness of graph representations obtained by different GNN models, we visualize the final node embeddings of the converged GNNs in Figure 5 and calculate the Pearson correlation coefficient with the target delay distribution across the entire circuit graph. The results show that our model can achieve the highest correlation score compared to other baselines, further validating its superiority.
Table 2: Mean Average Error of Prediction Results in Synthetic Dataset
Layers	Delay Prediction Error (ps)
Baselines	Ours
GCN	GAT	GraphSAGE	DeepGCN	GraphTrans	NTREE	SyncTREE	SyncTREE+TC	SyncTREE+C
4	12.675	11.192	9.855	9.231	9.077	9.457	8.745	8.622	8.699
8	13.037	9.834	9.798	9.193	9.283	9.432	6.631	6.521	6.590
16	13.338	7.791	9.732	9.438	7.853	8.461	3.775	3.648	3.912
32	13.353	6.942	10.278	9.363	9.518	8.217	3.424	3.354	3.370
64	13.782	13.356	10.232	9.178	8.859	7.022	3.556	3.451	3.523
Layers	Slew Prediction Error (ps)
Baselines	Ours
GCN	GAT	GraphSAGE	DeepGCN	GraphTrans	NTREE	SyncTREE	SyncTREE+TC	SyncTREE+C
4	37.548	33.186	32.705	29.561	29.121	29.781	25.672	25.579	25.601
8	39.004	35.164	33.429	29.338	29.483	29.772	20.540	20.215	20.620
16	40.348	27.817	34.908	29.691	28.352	28.368	14.305	14.047	14.285
32	41.827	25.847	34.857	29.563	27.446	28.581	12.864	12.608	13.107
64	43.913	42.092	36.962	29.758	28.897	27.044	14.544	14.426	14.591
Table 3: Mean Average Error of Prediction Results in RISC-V dataset
Layers	Delay Prediction Error (ps)
Baselines	Ours
GCN	GAT	GraphSAGE	DeepGCN	GraphTrans	NTREE	SyncTREE	SyncTREE+TC	SyncTREE+C
4	0.0467	0.0424	0.0417	0.0430	0.0359	0.0422	0.0313	0.0306	0.0317
8	0.0403	0.0403	0.0395	0.0372	0.0341	0.0298	0.0195	0.0182	0.0223
16	0.0343	0.0285	0.0371	0.0453	0.0319	0.0263	0.0128	0.0114	0.0131
32	0.0321	0.0304	0.0405	0.0311	0.0338	0.0271	0.0106	0.0098	0.0109
64	0.0379	0.0389	0.0436	0.0450	0.0357	0.0314	0.0176	0.0159	0.0181
Layers	Slew Prediction Error (ps)
Baselines	Ours
GCN	GAT	GraphSAGE	DeepGCN	GraphTrans	NTREE	SyncTREE	SyncTREE+TC	SyncTREE+C
4	0.5453	0.4739	0.5319	0.1514	0.1312	0.2231	0.0410	0.0406	0.0418
8	0.3719	0.3740	0.4726	0.1864	0.1436	0.1974	0.0265	0.0237	0.0258
16	0.3141	0.3257	0.3315	0.1908	0.1377	0.2085	0.0252	0.0228	0.0249
32	0.3004	0.2989	0.3569	0.1190	0.1349	0.1859	0.0464	0.0428	0.0491
64	0.2944	0.3101	0.4021	0.1875	0.1510	0.1922	0.0506	0.0501	0.0533
(a) MAE on S
(b) Percent error on S
Figure 3: Delay Prediction Result of SyncTREE on Synthetic dataset (S) and RISC-V dataset (R).
(c) MAE on R
(d) Percent error on R
(b) Percent error on S
(c) MAE on R
Figure 4: Slew Prediction Result of SyncTREE on Synthetic dataset (S) and RISC-V dataset (R).
Accuracy vs. circuit size and the number of sinks. To provide more insights into the performance of our method, we investigate the relationship between prediction accuracy and two important factors: circuit size and the number of sinks (i.e., leaf nodes) in RC trees. As depicted in Figure 3 and Figure 4, our method achieves better accuracy on circuits with larger sizes and more sinks.
TContrast loss vs. Circle loss. We provide the prediction results of SyncTREE models which trained with TContrast (TC) loss and Circle (C) loss in Table 2 and Table 3, we found that SyncTREE+TC performs better than the default L1 loss and Circle loss, highlighting the advantages of TC in optimizing similarity pairs and loss optimization convergence. To further compare the original
(b) Ours(0.94)
(c) GAT(0.82)
(d) NTREE(0.67)
(e) GraphTrans(0.41)
Figure 5: Visualization of final feature embeddings obtained by different GNNs.
(a) Circle Loss
Figure 6: Visualization of the similarity distributions after convergence.
(b) TContrast loss
Circle loss and TContrast loss, we analyzed their convergence status in the delay prediction task. Figure 6 shows the distribution of similarity pairs  $(s_n,s_p)$  after convergence. We observed that the similarity pairs using TContrast loss are closer to the optimization goal and have a more concentrated distribution in similarity space.
Computational Efficiency. SPICE typically requires significant computational resources to model circuit behavior in a time-incremental manner. However, considering that RC circuits are linear systems, we can utilize linear multi-step numerical integration techniques like the Trapezoidal method [31] to solve the differential-algebraic equations if the form of  $\frac{dx}{dt} = Ax$  inside SPICE for transient simulation. This results would be  $x(t + h) = (I - \frac{1}{2} hA)^{-1}(I + \frac{1}{2} hA)x(t)$ , given a circuit with  $n$  nodes and a driving signal with  $m$  time steps from the source. With LU factorization being used to invert  $n \times n$  matrices, the computation complexity of SPICE is expressed as  $O(m \cdot (n^3 + n^2 + n))$ .
In contrast, the computational complexity of our SyncTREE model is significantly reduced, which makes it highly efficient. As it's GAT-based, there are no extensive matrix operations required, and the computations can be parallelized easily. Moreover, the number of layers  $l$  is less than  $n$ . Thus, the time complexity of our model can be expressed as  $O(l \cdot (n \cdot f \cdot f' + (n - 1) \cdot f'))$ , where  $f$  is the number of input features,  $f'$  is the number of output features, and  $n - 1$  is the number of edges. We provide the computational efficiency comparison of SyncTREE with SPICE simulation in Figure 7. Our results indicate that SyncTREE is significantly faster than SPICE in obtaining accurate results, with the advantage increasing as the circuit size grows.
Transfer Learning. We provide the results of ablation study of transfer learning of SyncTREE on both delay and slew tasks in Table 4. During transfer learning, we froze the weights pre-trained on the other task and only fine-tuned the readout module on the target task. The results show that the representations learned from the delay prediction task can be efficiently transferred to the slew task, resulting in better performance. The indicates that SyncTREE's learned representation can be efficiently transferred between tasks, which can save time and resources by avoiding the need for additional training on every new target task from scratch.
In this paper, we propose a novel GNN model to predict circuit timing with a much faster computation speed than the SPICE simulator. Furthermore, based on the RC tree's structural and physical properties, we devise a tree contrastive (TContrast) loss to guide the feature transformation during graph learning. The results show that our model reaches state-of-the-art performance compared
Figure 7: Runtime of SPICE simulation and SyncTREE model with circuit size.
Layers	MAE Change with Transfer Learning
Delay to Slew	Slew to Delay
Synthetic	RISC-V	Synthetic	RISC-V
4	0.917↓	0.0009↓	0.720↑	0.0007↑
8	1.598↓	0.0021↓	0.326↑	0.0012↑
16	2.304↓	0.0032↓	0.343↑	0.0026↑
32	1.876↓	0.0054↑	0.969↑	0.0018↑
64	2.068↓	0.0037↑	0.505↑	0.0034↑
Table 4: SyncTREE Transfer Experiments Between Delay Prediction and Slew Prediction. (↓ means "lower is better")
with prospective GNNs and outperforms the SPICE simulator in computation efficiency. Therefore, our SyncTREE model can better support fast incremental timing updates. Compared to SPICE's full-fledged simulation even for incremental circuit updates, our solution's incremental updates will be even more appealing to IC design optimization.
This work was supported, in part, by the SUNY-IBM AI Collaborative Research Alliance Project #22003, the SUNY Empire Innovation Program, and, by Advantest as part of the Graduate School "Intelligent Methods for Test and Reliability" (GS-IMTR) at the University of Stuttgart.
[1] Guyue Huang, Jingbo Hu, Yifan He, Jialong Liu, Mingyuan Ma, Zhaoyang Shen, Quejian Wu, Yuanfan Xu, Hengrui Zhang, Kai Zhong, et al. Machine learning for electronic design automation: A survey. ACM Transactions on Design Automation of Electronic Systems (TODAES), 26(5):1-46, 2021.
[2] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4-24, 2020.
[3] Haoxing Ren, Siddhartha Nath, Yanqing Zhang, Hao Chen, and Mingjie Liu. Why are graph neural networks effective for eda problems? In Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design, pages 1-8, 2022.
[4] Zizheng Guo, Mingjie Liu, Jiaqi Gu, Shuhan Zhang, David Z Pan, and Yibo Lin. A timing engine inspired graph neural network model for pre-routing slack prediction. In Proceedings of the 59th ACM/IEEE Design Automation Conference, pages 1207-1212, 2022.
[5] Mingjie Liu, Walker J Turner, George F Kokai, Brucek Khailany, David Z Pan, and Haoxing Ren. Parasitic-aware analog circuit sizing with graph neural networks and bayesian optimization. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1372-1377. IEEE, 2021.
[6] Daniela Sánchez Lopera, Lorenzo Servadei, Gamze Naz Kiprit, Souvik Hazra, Robert Wille, and Wolfgang Ecker. A survey of graph neural networks for electronic design automation. In 2021 ACM/IEEE 3rd Workshop on Machine Learning for CAD (MLCAD), pages 1-6. IEEE, 2021.
[7] Yifan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, and Yichen Wei. Circle loss: A unified perspective of pair similarity optimization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6398-6407, 2020.
[24] Yiyue Qian, Chunhui Zhang, Yiming Zhang, Qianlong Wen, Yanfang Ye, and Chuxu Zhang. Co-modality graph contrastive learning for imbalanced node classification. Advances in Neural Information Processing Systems, 35:15862-15874, 2022.
[25] Laurence W. Nagel and D.O. Pederson. Spice (simulation program with integrated circuit emphasis). Technical Report UCB/ERL M382, EECS Department, University of California, Berkeley, Apr 1973.
[26] Chung-Wen Ho, Albert Ruehli, and Pierce Brennan. The modified nodal approach to network analysis. IEEE Transactions on circuits and systems, 22(6):504-509, 1975.
[27] Petar Velicković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.
[28] Jorge Rubinstein, Paul Penfield, and Mark A Horowitz. Signal delay in rc tree networks. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2(3):202-211, 1983.
[29] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
[30] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
[31] Lloyd N Trefethen and JAC Weideman. The exponentially convergent trapezoidal rule. siam REVIEW, 56(3):385-458, 2014.
A.1 Message Passing in SyncTREE
As the message passing process in Figure 8, information from leaves, sub-branches, and the whole global structure is first collected following the bottom-up propagation by  $\mathrm{GAT}_{bu}$ . Then, the final node representations of  $\mathrm{GAT}_{bu}$  are applied to each layer of  $\mathrm{GAT}_{td}$  to jointly update the node attributes of the corresponding top-down tree with the node embeddings of its previous layer. As the example shown in Figure 8, by designing this two-pass message-passing mechanism, the node features will incorporate the information from different levels and become more expressive. Furthermore, in the top-down tree, the root node can only be updated with synchronized  $h_{bu}^{L}$  since it doesn't have any incoming connection, it ensures that information injection at the source of the top-down tree is fixed which can help to maintain differentiable feature embeddings without over-smoothing.
Figure 8: Illustration of our two-pass message-passing mechanism.
A.2 Synthetic and RISC-V Dataset Preparation
Our dataset is composed of artificially generated and practical RC trees and the golden timing results at sinks (leaf nodes of each RC tree) obtained by SPICE simulation. On the one hand, we follow the pipeline in Figure 9 to generate the synthetic dataset. To be specific, we first adopt Algorithm 1 to generate RC-trees with random typologies and then convert them to artificial IC interconnects for further SPICE timing measurement. On the other hand, we directly extract RC trees from practical RISC-V circuit designs to compose the RISC-V dataset.
Figure 9: Pipeline for the synthetic dataset generation.
A.3 Baselines's Implementation
The GNNs of all the baseline models are set with 32/128 hidden dimensions separately for the synthetic dataset/RISC-V dataset. For GraphTrans, the dimension of the feedforward full-connection layers in the Transformer of GraphTrans is set to 256 with 0.1 dropout probability between layers, the number of attention heads is set to 4, and the max input sequence length is set to the maximum circuit size. It should be noted that we only made a little modification to the GraphTrans model. GraphTrans
Algorithm 1 Generate artificial RC-trees
Require:  $v_{d} \in [v_{min}, v_{max}]$ ,  $R \in [R_{min}, R_{max}]$ ,  $C \in [C_{min}, C_{max}]$   
Initialize voltage  $v_{d}$  of driving cell, edge type (rising or falling), depth  $D$  of RC tree  
parent set = list[drivingcell]  
parent ← randomly pick one element from parent set  
while depth ≤ D do  
    randomly choose R, C  
    generate child, add child into parent set  
    the  $R_{child}$  of the edge from child to the parent ← R  
    the  $C_{child}$  of child to the ground ← C  
    parent ← randomly pick one element from parent set  
     $D = D + 1$   
end while
is originally designed for node classification tasks, it takes CLS token from Transformer output as the representation of the whole graph and applies a linear module followed by softmax to implement prediction. In order to incorporate global information into node features, in our experiments, we concatenate the CLS token with node embeddings and then feed it into MLP to get the final output. For NTREE, we set GAT as its basic block with a 0.2 dropout probability between layers. We follow the original junction-tree-based algorithm in [10] to compose H-trees from our RC circuits with the same radius setting for extracting subgraphs in the paper.
A.4 Analysis of TContrast Loss To visualize the converging process during training, we plot the distribution of similarity pairs in space at different epochs in Figure 10. It obviously shows that our model approaches the optimization goal with a more concentrated similarity distribution after enough training with the guidance of TContrast loss. In Figure 11, we show the MAE difference of timing results obtained by vanilla SyncTREE and TC-loss guided SyncTREE. As shown in the results, after being combined with TC loss, our SyncTREE model has smaller errors for most types of RC trees which can effectively prove the validity of TC loss.
(a) Epoch  $= 1$
(b) Epoch  $= 20$
Figure 10: The distribution of similarity pairs with training epochs.
(c) Epoch  $= 40$
(d) Epoch  $= 60$
(a) Delay on S
(b) Slew on S
(c) Delay on R
Figure 11: Mean Average Error difference after applying TContrast loss on the synthetic dataset (S) and RISC-V dataset (R). (Negative values indicate that TC-loss-guided SyncTREE has a lower MAE error than vanilla SyncTREE)
(d) Slew on R