SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 6: GCX-ASTROG algorithm: The process starts with sampling and labeling a small set of points  $(\alpha)$ . A novel sampling criterion guides the algorithm towards a specific feature space region. The top  $\lambda$  points create a population (P) for the DE algorithm, generating  $m$  unlabeled points at every iteration of ASTROG. A semi-supervised learning framework uses  $\lambda + m$  points to learn a graph  $G(V, E, W, X_u, X_l, Y_l)$ . A pre-trained GNN propagates labels over the  $m$  points at each iteration. The best solution is integrated into the original dataset, and the process continues until the iteration budget is met.
to achieving optimal design outcomes. See Fig. 6 for the algorithm flow, and the step-by-step overview is as follows:
Initially, a dataset  $(D)$  is generated with  $\alpha$  samples such that  $D \in \mathbb{R}^{\alpha \times d}$ . The initial dataset is created using SPICE simulations within the bounds, denoted as  $a_i$  and  $b_i$  for each design metric. Stopping criteria are predefined, and the optimal design is outputted if these criteria are met; otherwise, the process moves to the next step. In step three, all designs in the database are ranked using the adopted infill sampling criterion [2], and the  $\lambda$  best designs are selected. New bounds  $\tilde{X}_i = [\tilde{a}_i, \tilde{b}_i]$  are obtained based on these designs, where  $\tilde{X}_i \in \mathbb{R}^{\lambda \times d}$ . The Differential Evolution (DE) algorithm [4] is used to optimize the objective function by generating  $m$  child solutions from the population within the newly obtained bounds. Next, a graph is learned (with the previously discussed approach) by combining the features of best  $\lambda$  labeled points with the  $m$  unlabeled points obtained from the DE algorithm. The obtained graph is passed through the pre-trained GNN models and labels are predicted for the  $m$  unlabeled nodes. After the label prediction, the  $\lambda + m$  labeled points are re-ranked using the infill sampling criterion [2]. The optimal design and its corresponding performance metric (label) are then selected from the  $\lambda + m$  labeled points and incorporated into the original database. This process is repeated until the predetermined iteration budget for each circuit is exhausted.
In this section, we showcase the experimental results of our two proposed algorithms: EASCO and ASTROG. To evaluate the effectiveness of these algorithms, we chose Two-stage OTA with Miller Compensation which is operating mostly in a linear region, and the Ring Oscillator (Three-stage) which is operating mostly in a nonlinear region as our test cases. The Miller Compensated OTA has 11 design variables, while the Ring Oscillator has 6 design variables. The performance evaluation of our algorithms was conducted using multiple metrics, which will be discussed in detail later. Our experimental workflow comprises four stages. We initiate with a comprehensive description of dataset generation using SPICE simulation and subsequently provide an exhaustive analysis of the algorithm's operational efficiency.
6.1 Harnessing EDA tools- SPICE simulator for Dataset generation
We have considered two test cases belonging to different families of circuits: Amplifiers and Oscillators. Design space bounds, Performance metrics and Specifications are carefully considered after designer's recommendation (mentioned subsequently). Features (design space points)  $X = (X_{u},X_{l})$  are generated by uniformly sampling points across the  $d$ -dimensional space forming a matrix of size  $X\in \mathbb{R}^{n\times d}$ . Since we employ semi-supervised learning we simulate labels corresponding to only  $l$  samples where  $l < n$  (samples are randomly chosen to avoid bias). The dataset obtained is :  $(X_{l},X_{u},Y_{l})$ .
Two-stage Miller Compensated OTA: Design parameters  $(X_{i})$  are: Reference current  $(I_{ref})$ , Reference Voltage  $(V_{dd})$ , Transistor widths and length  $(W_{1-8}, L_{1-8})$ , Capacitance  $(C_{c}, C_{L})$  and load resistance  $(r_{L})$ . Performance metrics  $(Y_{i})$  are: Gain, Unity Gain Bandwidth (UGB), Gain Margin (GM), Phase Margin (PM), Noise and Power.
Three-stage Ring Oscillator: Design parameters  $(X_{i})$  are: Reference Voltage  $(V_{dd})$ , Transistor widths and length  $(W_{1,2}, L_{1,2})$  and Capacitance  $(C_1)$ . Performance metrics  $(Y_{i})$  are: Frequency, rms Jitter, Delay and Power.
6.2 Performance evaluation of GCX surrogate model
To evaluate the efficacy of our proposed approach of learning graph of circuits, we conducted tests by integrating it with various GNN architectures [11, 7, 27]. For each test case we learned a graph with  $n = 1\mathrm{k}$  nodes and training was performed under different labeled data settings, with  $l$  being set to  $30\%$  and  $50\%$ . We compared the results of our proposed node-level regression approach with benchmark method ParaGraph [22], which is a graph-level regression setting, and ESSAB [2], a novel two-layer neural network for regression. The methods used for comparison do not facilitate semi-supervised learning hence experimental setup was adopted as in mentioned in [22, 2].
We implemented the graph formation technique described in ParaGraph [22] for our selected test cases. We established a graph level regression problem consisting of 15k graphs along with their corresponding label vectors. Additionally, we constructed a GNN architecture as outlined in the paper. However, during experimentation, we observed that the resulting  $R^2$  scores were predominantly negative. Consequently, we opted not to showcase the results for the surrogate model. We attribute this failure to two main factors: (i) The scarcity of node and edge features specific to the circuit under study. The ParaGraph [22] method was initially designed for industrial circuits with longer feature vectors, which might have impacted the performance on our dataset. (ii) The limited number of graphs available for training. Although the training loss initially decreased, it quickly reached saturation, leading to ambiguous predictions. Considering these factors, we believe that a more comprehensive feature set and a larger training dataset would be necessary to achieve more accurate results with the proposed approach.
We utilized the ESSAB surrogate model [2] with a dataset of 1k labeled samples. To ensure consistency in training conditions, we maintained a  $50\%$  training-to-testing ratio, similar to our approach.
Miller Compensated Two-stage OTA (R2 scores)
GCX(.)	Gain (dB)	UGB (MHz)	GM (dB)	PM (deg)	Noise (μV)	Power (μW)
l=30%	l=50%	l=30%	l=50%	l=30%	l=50%	l=30%	l=50%	l=30%	l=50%	l=30%	l=50%
(GCN)	0.17	0.22	0.36	0.43	0.52	0.81	0.70	0.81	0.35	0.64	0.42	0.61
(SAGE)	0.42	0.60	0.43	0.60	0.89	0.91	0.93	0.94	0.96	0.97	0.97	0.99
(GAT)	0.19	0.41	0.33	0.41	0.89	0.91	0.79	0.85	0.41	0.61	0.60	0.71
Table 1:  ${R}^{2}$  scores with different GNN architectures (Included value is the mean,averaged across 5 runs)
Three-stage Ring Oscillator (R2scores)
GCX(.)	Frequency (MHz)	Delay (μS)	Power (μW)
l=30%	l=50%	l=30%	l=50%	l=30%	l=50%
(GCN)	0.30	0.44	0.28	0.55	0.35	0.63
(SAGE)	0.45	0.83	0.56	0.82	0.84	0.91
(GAT)	0.41	0.50	0.34	0.58	0.36	0.58
Table 2:  ${R}^{2}$  scores with different GNN architectures (Included value is the mean,averaged across 5 runs)
Model	Gain	UGB	GM	PM	Noise	Power	Frequency	Delay	Power
GCX(SAGE)	0.60	0.60	0.91	0.94	0.97	0.99	0.83	0.82	0.91
ESSAB [2]	0.57	0.50	0.81	0.93	0.89	0.89	0.89	0.71	0.88
Table 3:  ${R}^{2}$  scores comparison between our best model GCX(SAGE) and ESSAB [2] surrogate model.
Our proposed approach, based on the semi-supervised learning framework, outperforms competing methods even when provided with minimal labeled data. We demonstrate the flexibility of our approach by evaluating it on two test cases representing different classes of circuits. Furthermore, we obtain a highly accurate graph-based surrogate model that will be utilized in the subsequent algorithms proposed in this work. Results of our approach are presented in Tables:1,2,3 respectively.
6.3 Comparative analysis between different Optimization strategies
Our approach involves first obtaining a highly accurate surrogate model, which we subsequently employ in two algorithms, as discussed earlier, to determine the optimal parameters for the given design problem. To evaluate the efficacy of our proposed algorithms, we compare them against two benchmark optimization methods - the DE algorithm [24] and the BO-EI algorithm proposed in [1]. To ensure reproducibility, we run each algorithm ten times. We then solve a single objective constrained optimization problem, based on the designer's recommendations, for each circuit, and report the obtained results. We selected the Figure of Merit (FOM) as the objective function to be minimized in our study. To define the FOM, we extensively reviewed the literature [19, 30] and derived a customized expression that aligns with our specific requirements. Further details regarding the formulation of this FOM can be found in the supplementary materials. The FOM expression we developed effectively balances the trade-off between different specifications, enabling us to optimize the input parameters for the given test cases. The best-case results are depicted in blue, while the second-best results are highlighted in black.
Miller Compensated Two-stage OTA (Optimization)
Objective: Minimize FOM st. Gain > 50 dB; UGB > 100 Mhz; PM > 45 deg; GM > 15 dB; Power < 900 μW; Noise < 600 nVrms
Model	Gain (dB)	UGB (MHz)	GM (dB)	PM (deg)	Noise (nV)	Power (μW)	Success (out of 10)
(specs)	Max	Min	Max	Min	Max	Min	Max	Min	Max	Min	Max	Min
DE [24]	58.6	53.3	190	165	19.2	17.8	46.7	45.0	478	463	815	633	04/10
BO-EI[1]	68.3	65.6	225	190	18.9	17.8	52.0	45.6	470	462	627	505	03/10
EASCO	67.3	64.4	215	192	19.2	18.5	47.2	45.5	470	462	722	490	04/10
ASTROG	80	61.6	145	106	26	23.5	52.5	47.9	497	446	716	318	06/10
ASTROG stands out as the undisputed champion among all competing algorithms. The convergence speed achieved by ASTROG is approximately  $8\mathbf{x}$  faster compared to the BO-EI algorithm [1]. The results are as shown in Table:4.
Table 4: Comparison table for different optimization strategies [24, 1] with EASCO and ASTROG
Three-stage Ring Oscillator (Optimization)
Objective: Minimize FOM st. Freq > 1000 MHz; Jitter < 10 pS; Delay < 200 pS; Power < 600 μW
Model	Frequency (MHz)	Jitter(pS)	Delay(pS)	Power(μW)	Success
(specs)	Max	Min	Max	Min	Max	Min	Max	Min	(out of 10)
DE [24]	1265	1005	5.07	0.99	165	131	302	256	02/10
BO-EI[1]	2103	1047	2.73	10-7	159	79	407	66.4	06/10
EASCO	1444	1396	1.7	10-7	119	115	360	331	09/10
ASTROG	2033	1175	6.05	0.2	141	81.9	502	182	04/10
Table 5: Comparison table for different optimization strategies [24, 1] with EASCO and ASTROG
Among the competing algorithms, BO-EI [1] proved to be a formidable rival for our proposed algorithms. Although EASCO maintained its higher rank in terms of reproducibility, ASTROG delivered the second best results. However, the key factor in determining the champion algorithm lies in the speed of convergence towards the optimal value. In this regard, ASTROG's impressive  $4\mathbf{x}$  faster convergence clearly establishes it as the ultimate winner. The results are as shown in Table:5. For a comprehensive analysis of the timing results for each algorithm, please refer to the supplementary materials.
6.4 Knowledge Transfer using Zero-shot and Few-shot learning
To broaden the scope of our approach, we incorporated different technology nodes and topologies into our study. We selected a Three-stage ring oscillator as our initial test case and implemented the procedure for creating a comprehensive feature vector. We specifically focused on conducting few-shot (FS) learning experiments using  $180\mathrm{nm}$  technology nodes. We then assessed the generalization of the trained model to  $65\mathrm{nm}$  technology nodes and a five-stage topology. Additionally, we performed zero-shot (ZS) learning experiments using  $45\mathrm{nm}$  technology nodes. This allowed us to evaluate the model's performance without any prior training on the specific technology node.
Ring Oscillator (Three and Five-stage, 180 nm, 65 nm, 45 nm)
Model	3-S,FS(65 nm)	3-S,ZS(45 nm)	3to5-S,FS(65 nm)	3to5-S,ZS(45 nm)
l=30%	l=0%	l=30%	l=0%
SAGE	0.41	0.09	0.28	0.04
Table 6: Few-shot and Zero-shot generalization over technology and topology
In our experimental setup, we focused on utilizing a limited number of labeled samples to address the challenge of computational burden associated with generating labeled data. We considered a total of  $n = 1\mathrm{k}$  labeled samples and evaluated the performance based on the percentage of data  $(l)$  representing the infusion of technology/topology information for both few-shot and zero-shot learning. Given the constraints on labeled data availability, zero-shot learning was found to be ineffective in achieving satisfactory performance. However, there was some promising progress observed with few-shot learning, which indicates the potential of utilizing a small amount of labeled data to enhance the learning process.
Our new method for creating graph of circuits produced a consistent, homogeneous graph that can be used across multiple circuits. This approach lead to an accurate surrogate model with a computationally inexpensive node-level regression problem. By utilizing semi-supervised learning algorithms, we significantly reduced reliance on labeled datasets. The proposed GCX(GNN) surrogate model was further incorporated into EASCO and ASTROG. For the above mentioned test cases these algorithms outperformed the competing state-of-the-art optimization methods and yielded the most optimal design parameters that precisely meets the designer's criteria.
We extend our gratitude to the reviewers for their valuable comments and insightful suggestions, which have significantly enhanced the quality of this paper. We also appreciate the contributions of the entire MISN lab community for their engaging discussions related to this work. Our research has been made possible through the support of the DST Inspire faculty grant MI02322G.
[1] Shady A. Abdelaal, Ahmed Hussein, and Hassan Mostafa. A bayesian optimization framework for analog circuits optimization. In 2020 15th International Conference on Computer Engineering and Systems (ICCES), pages 1-4, 2020.
[2] Ahmet Faruk Budak et al. An efficient analog circuit sizing method based on machine learning assisted global optimization. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 41(5):1209-1221, 2022.
[3] M.T.M. Emmerich et al. Single- and multiobjective evolutionary optimization assisted by gaussian random field metamodels. IEEE Transactions on Evolutionary Computation, 10(4):421-439, 2006.
[4] Manolis Georgioudakis and Vagelis Plevris. "a comparative study of differential evolution variants in constrained structural optimization.", (2020) Accessed November 21, 2022.
[5] Kourosh Hakhamaneshi, Marcel Nassar, Mariano Phipell, Pieter Abbeel, and Vladimir Stojanovic. Pretraining graph neural networks for few-shot analog circuit modeling and design, 2022.
[6] Kourosh Hakhamaneshi, Nick Werblun, Pieter Abbeel, and Vladimir Stojanovic. Bagnet: Berkeley analog generator with layout optimizer boosted with deep neural networks, 2019.
[7] William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs, 2017.
[24] Samrat L. Sabat, K.Shravan Kumar, and Siba K. Udgata. Differential evolution and swarm intelligence techniques for analog circuit synthesis. In 2009 World Congress on Nature Biologically Inspired Computing (NaBIC), pages 469-474, 2009.
[25] Keertana Settaluri et al. Autockt: Deep reinforcement learning of analog circuit designs, 2020.
[26] Keertana Settaluri, Ameer Haj-Ali, Qijing Huang, Kourosh Hakhamaneshi, and Borivoje Nikolic. Autockt: Deep reinforcement learning of analog circuit designs, 2020.
[27] Petar Velivcković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks, 2017.
[28] Zhiyao Xie et al. Routenet: Routability prediction for mixed-size designs using convolutional neural network. In 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pages 1-8, 2018.
[29] Biying Xu, Yibo Lin, Xiyuan Tang, Shaolan Li, Linxiao Shen, Nan Sun, and David Z. Pan. Wellgan: Generative-adversarial-network-guided well generation for analog/mixed-signal circuit layout. In 2019 56th ACM/IEEE Design Automation Conference (DAC), pages 1-6, 2019.
[30] Khalil Yousef. A low phase noise, high figure of merit, 3.1 ghz-3.5 ghz ring oscillator using edge injection technique. 2017 Japan-Africa Conference on Electronics, Communications and Computers (JAC-ECC), pages 37-40, 2017.
[31] Kaixiong Zhou, Xiao Huang, Daochen Zha, Rui Chen, Li Li, Soo-Hyun Choi, and Xia Hu. Dirichlet energy constrained learning for deep graph neural networks. Advances in Neural Information Processing Systems, 34:21834-21846, 2021.