SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Aditya Hemant Shahane<sup>1</sup>
Swapna Manjiri
Ankesh Jain
Sandeep Kumar $^{1,2,3}$
<sup>1</sup>Bharti School of Telecommunication Technology and Management
3 Yardi School of Artificial Intelligence
Indian Institute of Technology Delhi
2Electrical Engineering
{bsy217530,een212025,ankesh,ksandeep}@iitd.ac.in
The design automation of analog circuits poses significant challenges in terms of the large design space, complex interdependencies between circuit specifications, and resource-intensive simulations. To address these challenges, this paper presents an innovative framework called the Graph of Circuits Explorer (GCX). Leveraging graph structure learning along with graph neural networks, GCX enables the creation of a surrogate model that facilitates efficient exploration of the optimal design space within a semi-supervised learning framework which reduces the need for large labeled datasets. The proposed approach comprises three key stages. First, we learn the geometric representation of circuits and enrich it with technology information to create a comprehensive feature vector. Subsequently, integrating feature-based graph learning with few-shot and zero-shot learning enhances the generalizability in predictions for unseen circuits. Finally, we introduce two algorithms namely, EASCO and ASTROG which upon integration with GCX optimize the available samples to yield the optimal circuit configuration meeting the designer's criteria. The effectiveness of the proposed approach is demonstrated through simulated performance evaluation of various circuits, using derived parameters in  $180\mathrm{nm}$  CMOS technology. Furthermore, the generalizability of the approach is extended to higher-order topologies and different technology nodes such as  $65\mathrm{nm}$  and  $45\mathrm{nm}$  CMOS process nodes.
Designing Electronic Design Automation (EDA) tools for analog circuits is challenging as it involves optimizing multiple parameters and their non-linear dependencies. Machine learning models are increasingly being integrated within EDA tools aiming to make this process flexible and suited for analog design across different circuits and technology nodes. Instead of simulation-based global optimization [14], which can be time-consuming for stricter requirements [8], a surrogate-model-assisted optimization approach using less computationally expensive approximation models have been proposed.
Regarding surrogate models, Bayesian optimization and Gaussian process models have been more effective than standard artificial neural networks and radial basis functions. However, Gaussian process modeling is computationally expensive [3], and as circuit complexity increases, inter-sample relations become harder to capture, training time increases and generalization ability is lost. To address these challenges, graph-based models that integrate Graph neural networks (GNNs) [5, 22, 17, 29] have emerged as a powerful technique in high-level synthesis, behavior modeling [9], placement and routing [20, 18, 28, 13], analog and mixed-signal layout exploration [25], making them a plausible choice for graph-based surrogate models. Graphs capture both topological and feature-based information, resulting in faster modeling and optimization of complex systems. Despite their
success, graph-based methods still require improvement in terms of independence from labeled data and generalization ability across different circuit topologies and technology nodes. For optimization algorithms, previous works have employed reinforcement learning [26], and evolutionary algorithms [6]. However, the models or policies that are learned via these methods are highly tailored to the environment or the objective of the optimization and lack reproducibility. Another popular method is the particle swarm intelligence [23] however, it easily falls into local optima in the high dimensional space and suffers from low convergence rate.
This paper introduces a novel and innovative approach to optimize circuit design by leveraging graph representation and graph-based semi-supervised learning. The approach is designed to enhance accuracy and efficiency while reducing the reliance on extensive labeled datasets. To achieve this goal, a semi-supervised learning framework is employed for the graph-based surrogate model. The accuracy of the approach is compared against the benchmark model ParaGraph [22], and its superior performance is demonstrated. Additionally, a new method is introduced to create a comprehensive feature vector that integrates information about various technology nodes and topologies, emphasizing generalizability in zero-shot and few-shot learning frameworks. By integrating these approaches, two new optimization methods on graph-based surrogate models are proposed: Efficient Analog Sizing via Constrained Optimization (EASCO) and Analog Sizing through Real-time Online Graphs (ASTROG). The paper is structured into seven sections, covering graphs, graph-based techniques, optimization strategies, the proposed approach, simulation results, and concluding remarks.
Construction of a graph-based surrogate model starts with circuit representation as graphs and the creation of a labeled graph dataset. The task is formulated as a graph-level regression problem with each graph mapped to its corresponding label vector (performance metric) using Graph Neural Networks (GNNs).
Graph representation for Analog and Mixed-signal circuits A graph is a data structure used to store semantic information with the help of its unique structure formed using nodes and edges. A graph with features is denoted by  $G = (V, E, W, X, Y)$  where  $V = \{v_1, v_2, \ldots, v_n\}$  is the vertex set,  $E \subseteq V \times V$  is the edge set and  $W$  is the adjacency (weight) matrix. We consider a simple undirected graph without self-loop:  $W_{ij} > 0$ , if  $(i,j) \in E$  and  $W_{ij} = 0$ , if  $(i,j) \notin E$ . Finally,  $X \in \mathbb{R}^{n \times d} = [X_1, X_2, \ldots, X_n]^T$  and  $Y \in \mathbb{R}^{n \times m} = [Y_1, Y_2, \ldots, Y_n]^T$  are the feature matrix and label matrix respectively, where the row vectors  $X_i \in \mathbb{R}^d$  and  $Y_i \in \mathbb{R}^m$  are the feature vector and label vector, respectively associated with one of  $n$  nodes of the graph  $G$ . The most common approach
Figure 1: The figure (a) depicts two distinct methods for representing circuits as a graph. In [22] circuit components are depicted as nodes while nets carry edge-based information. To this, [5] suggested explicitly representing device terminals as nodes to avoid ambiguity when the device has identical net connections between two or more terminals. Meanwhile, in figure (b), we can see the training framework of ParaGraph [22] method. For the example circuit used, multiple graphs and corresponding label vectors are used for training, and prediction on the unlabeled graph is performed.
for creating a circuit graph is based on the circuit's topological design. In this approach, circuit components such as resistors, capacitors, and transistors are depicted as nodes, while the circuit wiring (net) is mapped using edge-based information that connects different nodes [22]. This approach results in a heterogeneous graph that is useful for analyzing and developing complex electronic systems. As a result, a graph-level prediction problem [5, 22, 17] arises where we have a set of  $n$  graphs denoted by  $\{G_1, G_2, \ldots, G_n\}$ , each with its own set of nodes and edges, a function  $f$  which
maps these graphs to corresponding prediction metric  $(Y_{i})$ , where  $Y_{i} = f(G_{i})$ . Let  $X_{i}$  be the feature matrix for  $i^{th}$  graph where  $X_{i} \in R^{n \times F}$  where  $n$  is the number of nodes in a single graph and  $F$  is the features of each node. Overall, the problem of graph-level prediction using GNNs involves learning the function  $f$  based on graph structure and node features that map the input graphs to the desired prediction metric. Building upon the previously presented method for representing circuits as a graph, [5] proposed a refined approach by explicitly including device terminals as nodes in the graph. This modification results in featureless edges.
While this approach offers a detailed understanding of the interconnection between different circuit components, it suffers from several limitations: (i) As the circuit size increases, the graph becomes densely interconnected, leading to a large number of nodes and edges, making it challenging to comprehend the overall behavior of the circuit. (ii) Creating a large labeled graph dataset is time-consuming and computationally expensive, which hinders effective GNN training. (iii) This framework lacks inter-graph connections, which reduces its reliability. (iv) Since graph regression problems cannot be applied to semi-supervised frameworks, the reliance on labeled datasets is inevitable. (v) Additionally, training and validating over multiple graphs  $(n)$  with  $N$  nodes and  $E$  edges when passed through  $L$  layer GNN makes the process slow, and the high time-complexity  $(O(n * LN^2 E))$  and space-complexity  $(O(n * N^2 L))$  further adds to the computational burden. (vi) Finally, transferring knowledge across different topologies becomes challenging due to varying graph sizes.
3 Proposed Formulation and Background
For our case, consider  $n$  number of circuit instances indexed as  $i = 1,\dots ,n$ , where  $X_{i}\in \mathbb{R}^{d}$  be the feature vector and  $Y_{i}\in \mathbb{R}^{m}$  the label vector corresponding to  $i$ -th circuit. A feature vector  $X_{i}$  contains the circuit-level parameters e.g. resistance, capacitance, transistor widths, lengths, junction voltages etc and the corresponding label vector  $Y_{i}$  contains performance parameters, including Gain, Bandwidth, Noise, etc. The problem under consideration is:
Given  $n = l + u$  circuit instances and their parameters  $X = (X_{l},X_{u})$  along with labels of only  $l$  number of circuit instances as  $Y_{l}$ , how can we predict the labels of the remaining  $u$  number of circuit instances  $Y_{u}$ ?
The circuit performance parameters or the labels  $Y_{i}$  are generally obtained using expensive SPICE simulation, thus having a large set of labeled data is a luxury. Limited labeled data where  $l < u$  is a significant hurdle in learning models for EDA. Graph-based Semi-supervised learning attempts to address this by learning from both labeled as well as unlabeled data.
Consider sets  $T_{l} = \{(X_{i},Y_{i})\}_{i = 1}^{l}\in \mathcal{X}\times \mathcal{Y}$  and  $T_{u} = \{(X_{i})\}_{i = l + 1}^{l + u}\in \mathcal{X}$ , where  $\mathcal{X}$  is the domain set and  $\mathcal{V}$  is the label set. Suppose the elements of the set  $T_{l}$  be sampled from probability distribution  $\mathbb{P}$  over the ordered pair  $\mathcal{X}\times \mathcal{Y}$ . Samples of the domain set  $X_{i}\in \mathcal{X}$  will follow the marginal distribution  $\mathbb{P}_{\mathcal{X}}$  of  $\mathbb{P}$ . For the limited labeled data scenario, the goal of a learning algorithm is to estimate conditional distribution  $\mathbb{P}(Y|X)$  using both the labeled and unlabeled data as the training data. To incorporate the additional information of  $\mathbb{P}_{\mathcal{X}}$  obtained because of the unlabeled data by capturing the intrinsic geometry of the marginal distribution through the graph of both unlabeled and labeled data and performing label propagation and feature aggregation over the neighborhood. The guiding principle is that if two data points from the domain set are close in the intrinsic distribution  $\mathbb{P}_{\mathcal{X}}$  then the corresponding conditional distributions should be similar. For the circuit prediction, it means that given two circuit instances with similar features  $X_{i}\sim X_{j}$  then the corresponding label vectors should also be similar  $Y_{i}\sim Y_{j}$ .
To build a learning framework based on the above guiding principle, these are the two main steps i) First use the feature space to obtain a geometric representation such that the samples having similar features are represented closer and connected ii) Build a learning model such that geometric information be leveraged for learning the labels of unknown samples.
3.1 Proposed Framework for Semi-Supervised Learning on Graph of Circuits
We introduce a graph-based semi-supervised learning technique for building a surrogate model (GCX) for analog circuit simulation with limited labels. The proposed framework consists of four main stages: i) Firstly for obtaining feature-aware geometrical representation we construct a graph of circuits using circuit features  $X = (X_{l},X_{u})$  in the form of an adjacency matrix  $A$ , where each circuit
$i$  is a node of the graph and the circuit-level parameters, such as resistance, capacitance, transistor widths and lengths, and junction voltages, as its corresponding feature vector  $X_{i}\in \mathbb{R}^{k}$ . ii) Further using the graph adjacency matrix, feature matrix, and available set of circuit performance matrix together, i.e.,  $G(X,A,Y_l)$  with the graph neural network pipeline to build a machine learning model, iii) Constructing a comprehensive feature vector that facilitates generalizability (discussed later), iv) predicting the performance parameters  $Y_{u}$  corresponding to  $X_{u}^{\prime}s$ . Fig 2 illustrates the outline of the proposed framework:
Figure 2: Graph of Circuits Explorer (GCX): The original dataset (D) contains both labeled  $(X_{l},Y_{l})$  and unlabeled  $(X_{u})$  samples, each circuit is a node of the graph and sample as the feature vector of the associated node that will serve as nodes in the graph; the graph is learned (weighted adjacency-W) using the suggested formulation; and the learned graph is passed through a GNN model for label prediction for unlabeled nodes. This forms a GCX(GNN) surrogate model. For the above circuit  $X_{li} = (V_{dd},W1,L1,W2,L2,V_{in})$  and  $Y_{li} = Vo$ .
Algorithm 1 GCX(GNN)
Input: Feature matrix  $(X_{l},X_{u})$ , label matrix  $(Y_{l})$
Output: Optimal design  $(X_{i}^{*})$
1.  $G(V,E,W)\longleftarrow$  Learning Graph of Circuits Using  $(X_{l},X_{u})$
2.  $Y_{u} \gets$  Label prediction of unlabeled circuits using GNN with  $G(V, E, W, X_{l}, X_{u}, Y_{l})$ ;
3.  $\mathrm{GCX}(\mathrm{GNN})\gets G(V,E,W,X_u,X_l,Y_l) + \mathrm{GNN}$ ; surrogate model
4.  $X_{i}^{*}\longleftarrow$  GCX(GNN) + EASCO/ASTROG
Algorithm 1, which we denote by GCX(GNN), summarizes the outline of the proposed framework. The implementation details of each step are discussed subsequently.
This approach allows us to incorporate multiple circuit instances into a single graph while accommodating different technology nodes and topological diversity. We achieve this while maintaining a homogeneous graph with node feature vectors of equal lengths, making the training of GNNs in a semi-supervised framework efficient and cost-effective. Compared to the previous approach that handles circuits as graphs, our approach with  $N$  nodes in a graph with  $E$  edges when passes through  $L$  layer GNN has an overall time complexity of  $O\left(L*N^{2}E\right)$  and space complexity of  $O\left(N^{2}L\right)$ .
3.2 How to Learn Graph of Circuits to Encode Circuit Similarity
Consider the feature matrix of circuit instances  $X = [X_{1},X_{2},\ldots ,X_{n}]^{T}$  where  $X_{i}$  is the feature of  $i^{th}$  circuit which corresponds to  $i^{th}$  node of a graph. In the context of modeling signals or features with graphs, the widely used assumption is that the signal residing on the graph changes smoothly between connected nodes [10]. For example, if two circuit instances  $X_{i}$  and  $X_{j}$  have similar features then their Euclidean distance  $\| X_{i} - X_{j}\|^{2}$  should be smaller but connecting edge weight which quantifies similarity i.e.  $w_{ij}$  should be large. The Dirichlet energy (DE) is used for quantifying the smoothness of the graph signals which is defined as:
$$
\mathrm {D E} = \frac {1}{2} \sum_ {i, j} w _ {i j} \| X _ {i} - X _ {j} \| ^ {2}
$$
The lower value of Dirichlet energy indicates a desirable configuration [31, 10]. Smooth graph signal methods are an extremely popular family of approaches for a variety of applications in machine learning and related domains [12, 10]. In order to build a graph of circuits we will use the smoothness
of signals on graph assumption. Nodes connected with stronger weights indicate similar features and performance indices, facilitating knowledge transfer to new circuit instances. Finally, we learn a weighted adjacency matrix  $W$  by minimizing the Dirichlet energy combined with other sparsity regularization terms which entails solving the following optimization problem:
$$
w := \arg \min _ {w \in w _ {m}} \frac {1}{2} \sum_ {i, j} w _ {i j} \left\| X _ {i} - X _ {j} \right\| ^ {2} - \alpha 1 ^ {\top} \log (S w) + \beta \| w \| _ {2} ^ {2}
$$
where  $\alpha > 0$  and  $\beta \geq 0$  controlling the properties of the resultant graph,  $\|w\|_2^2$  ensures the graph is sparse, while  $1^\top \log (Sw)$  ensures that every node will have some connections. This problem is a convex optimization problem and efficiently solvable [12, 10].
3.3 Comprehensive feature vector representation for GCX
To streamline GNN training across various technology nodes and topologies, we extend the initial feature vector with one-hot encoded bits, specifically allocated for the technology file and topology. This augmentation generates a new feature vector that facilitates relevant pattern learning by the GNN. This approach significantly reduces computational costs, making it more viable for practical applications. With this trained GNN model, it becomes possible to accurately predict the performance of circuits that are yet to be seen, thereby enhancing circuit design and optimization. Further information about our proposed method is available in Figure 3.
Figure 3: Knowledge Transfer: To enhance the feature vector, we append it with one-hot encoded technology files and circuit topology details, resulting in a new feature vector. The technology files used,  $180\mathrm{nm}$ ,  $65\mathrm{nm}$ , and  $45\mathrm{nm}$ , are represented using 3 bits. To represent the original and higher-order topology, 2 bits are used, with an additional  $k$  bits assigned for circuit element features. The first figure shows the ring oscillator implemented with the  $65\mathrm{nm}$  technology file, while the second figure displays the 5-stage topology of the circuit.
3.4 Graph Circuit Explorer-Graph Neural Networks: GCX(GNN) surrogate model
Figure 4: The figure (a) depicts message passing in GNNs: Graph is passed through the GNN architecture; nodes interact with each other via message passing and feature aggregation for target node is carried out. Unlabeled nodes gather information from its labeled neighborhood in form of embeddings. Embeddings updated over  $L$  layers is used for label prediction. To better understand GNN architectures, refer: [11, 7, 27]. Meanwhile, in figure (b), we can see schematics of (i) Two-stage OTA with Miller Compensation and (ii) Three stage Ring Oscillator on which the algorithms will be implemented.
After learning a graph using the feature vector description provided, the graph undergoes label propagation through the GNN architecture. In this process, each node represents a circuit instance and the graph consists of both labeled and unlabeled nodes. During each layer of the GNN, the nodes interact through message passing, and feature aggregation is carried out. The unlabeled circuit instance acquires information from its labeled neighbors in the form of embeddings. The embeddings
are iteratively updated across the  $L$  layers of the GNN, ultimately leading to the prediction of a label for the unlabeled circuit instance (node). The formulation for the embedding update can be understood from the equation given below. The process can be understood from Fig:4
$$
h _ {u} ^ {(k)} = \sigma \left(W _ {\text {s e l f}} ^ {(k)} h _ {u} ^ {(k - 1)} + W _ {\text {n e i g h}} ^ {(k)} \sum_ {v \in N _ {u}} h _ {v} ^ {(k - 1)} + b ^ {(k)}\right)
$$
$h_u^{(k-1)} \in \mathbb{R}^{d^{(k-1)}}$ : Node embeddings of target node  $u$  at layer  $k$ .  $W_{\mathrm{self}}^{(k)}, W_{\mathrm{neigh}}^{(k)} \in \mathbb{R}^{d^{(k)} \times d^{(k-1)}}$ : are the learnable parameters,  $b^{(k)} \in \mathbb{R}^{d^{(k)}}$ : represents the bias term and  $\sigma$ : represents elementwise non-linearity (e.g., a tanh or ReLU). GCX, the approach of creating a graph of circuits, seamlessly integrates with any GNN architecture, showcasing its flexibility across circuits and architectures. Extensive experiments provide conclusive evidence of its effectiveness.
4 Efficient Analog Sizing via Constrained Optimization (GCX-EASCO)
Figure 5: GCX-EASCO algorithm: Initially points are uniformly sampled and labels are generated based on the value of  $l$ ; graph  $G(V, E, W, X_u, X_l, Y_l)$  is learned using all the points based on features; GNNs in a semi-supervised framework learn the embeddings corresponding to labeled samples, which are used for label prediction corresponding to unlabeled nodes  $G(V, E, W, X, Y)$ . Ultimately, constrained optimization over the subset  $\hat{D}$  of the original dataset gives optimum parameters  $X_i^*$ .
Our new algorithm employs a more efficient graph-based surrogate model to obtain optimal circuit parameters. The algorithm uses a static, offline training framework that has been recommended in [15, 16]. The algorithmic flow is described in Fig:5 and the following is the step-by-step overview:
The EASCO algorithm consists of four main components. Firstly, in the data generation phase,  $n$  points are uniformly sampled from a  $d$ -dimensional feature space such that  $X_{i} \in [a_{i}, b_{i}]$  where  $a_{i}$  and  $b_{i}$  are the lower bound (LB) and upper bound (UB) for each feature. This creates a dataset (D) such that  $D \in \mathbb{R}^{n \times d}$ . Next, a graph is learned (as previously discussed) using all the  $n$  sampled points, with a subset of these points simulated under different labeled data settings ( $l$ ), such as  $30\%$ , and  $50\%$ , by adopting a semi-supervised learning framework. The labeled points,  $X_{train} \in \mathbb{R}^{l \times d}$  are used for training, while the rest,  $X_{test} \in \mathbb{R}^{(n - l) \times d}$  are reserved for testing. In the third component, GNN models [11, 7, 27] are used to make predictions. Lastly, in the constrained optimization component, uniform sampling and integration with label propagation over unlabeled samples offer an overall comprehension of the feature_search) space. With constraints in place, the process returns a subset of the original search space  $\hat{D} \in \mathbb{R}^{k \times d}$  where  $k << n$ . The objective function is optimized using Differential Evolution (DE) algorithm [4] applied to the resulting parameter space, yielding the most optimal parameters.
5 Analog Sizing Through Real-time Online Graphs (GCX-ASTROG)
The previous algorithm efficiently minimized simulations once coarse boundaries were established. However, handling an increasing number of specifications made it challenging to identify these boundaries. To overcome this, we developed an online training version of the algorithm that requires minimal initial data. It utilizes an optimization algorithm, a surrogate model, and an infill sampling criterion [21] to guide the search towards global optima. The algorithm updates iteratively and provides the optimal circuit design parameters upon meeting the stopping criterion. Our adoption of the ranking criterion from [2] ensures consideration of all constraints and enables a precise approach