AUTOCIRCUIT-RL: Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation
Authors: Prashanth Vijayaraghavan
Luyao Shi
Ehsan Degan
Vandana Mukherjee
Xin Zhang
Venue: ICML 2025 poster (2025)
Keywords: Analog circuit
circuit topology
reinforcement learning
instruction tuning
RL refinement
Primary area: applications->everything_else
TL;DR: We introduce AUTOCIRCUIT-RL, an AI system that uses reinforcement learning and large language models to automatically generate efficient and valid analog circuit designs from constraints.

Abstract: Analog circuit topology synthesis is integral to Electronic Design Automation (EDA), enabling the automated creation of circuit structures tailored to specific design requirements. However, the vast design search space and strict constraint adherence make efficient synthesis challenging. Leveraging the versatility of Large Language Models (LLMs), we propose AUTOCIRCUIT-RL, a novel reinforcement learning (RL)-based framework for automated analog circuit synthesis. The framework operates in two phases: instruction tuning, where an LLM learns to generate circuit topologies from structured prompts encoding design constraints, and RL refinement, which further improves the instruction-tuned model using reward models that evaluate validity, efficiency, and output voltage. The refined model is then used directly to generate topologies that satisfy the design constraints. Empirical results show that AUTOCIRCUIT-RL generates ~12% more valid circuits and improves efficiency by ~14% compared to the best baselines, while reducing duplicate generation rates by ~38%. It achieves over 60% success in synthesizing valid circuits with limited training data, demonstrating strong generalization. These findings highlight the framework's effectiveness in scaling to complex circuits while maintaining efficiency and constraint adherence, marking a significant advancement in AI-driven circuit design.

AUTOCIRCUIT-RL:
Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation
Prashanth Vijayaraghavan !
Luyao Shi" 1
Ehsan Degan
Vandana Mukherjee
Xin Zhang?
Abstract
component clrcuit Instraction/ Prrmpph: Generate
with inductor: inductor-oi
MOSFET::
PRT-A-0 and PET-A-t-ad MOSFET: FRT::FR--RE FRI-B-O and FFT-B- represeting different uodes l'inductor-0', TET-A-1, (PET-A-0', "FET-B-o", (FET-B-1
Analog circuit topology synthesis is integral to Electronic Design Automation (EDA), enabling the automated creation of circuit structures tai- lored to specific design requirements. However, the vast design search space and strict constraint adherence make efficient synthesis challenging. Leveraging the versatility of Large Language Models (LLMs), we propose AUTOCIRCUTT-RI, novel reinforcement learning based frame- work for automated analog circuit synthesis. The framework operates in phases: instruction tuning, where an LLM learns generate circuit topologies from structured prompts encoding de- sign constraints, and RL refinement, which fur- ther improves the instruction-tuned model using reward models that evaluate validity, efficiency, and output voltage. The refined model is then used directly to generate topologies that satisfy the design constraints. Empirical results show that AUTOCIRCUIT-RL generates ~12% more valid circuits and improves efficiency by 144 compared to the best baselines, while reducing du- plicate generation rates by ~38%. It achieves over 60% success in synthesizing valid circuits with limited training data, demonstrating strong gen- eralization. These findings highlight the frame- work's effectiveness in scaling to complex circuits while maintaining efficiency and constraint ad- herence, marking a significant advancement in Al-driven circuit design.
with effieiency with Yout less than 1gVV when VIN equals
than 0.77
AUTOCIRCUT-RL
Dabah comodedd drain
F-TA- Ietumted & m
1N
IndudtenCos FET-B
V1
IN
F-B-C
10
F-B-1
OUT
F-A-0
F-1-
Dads Cycle
e
NGSpice Simulator
Elficiecyy 0.8 Vout 0.45V
Figure 1. Given a prompt with design constraints, the goal to gen- erate circuit topology and duty cycle satisfying those constraints. Component constraints (blue) are mandatory, while efficiency (or- ange) and voltage (purple) constraints optional. In the output, blue circles denote components ("F" for FET, "T" for inductor), and yellow circles denote connection nodes.
1. Introduction
Al and machine learning have been applied to various circuit design tasks, including parameter optimization (Wang et al., 2020) and physical design (Hakhamaneshi et al., 2019), which focus on circuit optimization with a fixed circuit topology. Analog circuit topology synthesis (Bengio et al., 2013) is a fundamental aspect of EDA, where the configura- tion and interconnection of components directly influence circuit functionality and performance. Despite years of EDA advancements, automation of analog circuit topology synthesis has remained underexplored until recently.
'Equal contribution 'IBM Almaden Research Center, San Jose. CA 95120, USA 'IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598. Correspondence to: Prashanth Vijayaraghavan <prashanthv com>, Luyao Shi <luyao.shi sibmomsshiibb..mmm.... Xin zhang <xzhang @ us. ww..ibb.com......
The key challenge in circuit topology synthesis stems from the exponential growth the design space with the num- ber of components, making high-quality designs rare and
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

AUTOCIRCUIT RL: Reinforcement Learning-Drivenr LLM for Automated Circuit Topology Generation
hard discover. This sparsity makes it difficult satisfy specifie performance and design constraints. Manual topol- ogy design remains time-consuming and requires significant expertise, while brute-force or random search methods are computationally infeasible due to the vastness of the space.
Atelier (Shen et al., 2024) incorporate domain knowledge, subeircuit libraries, Chain-of- Thought prompting, and agent- based coordination. Nonetheless, these methods lack rein- forcement learning for iterative, performance-dreg ment, focusing instead on structured generation and error correction. Our work advances beyond prior efforts by synthesizing more complex circuits while optimizing both topology and performance metrics.
refine-
Existing AI methods fall into two categories: (a) Al-based search-based algorithms, including rule-based systems, heuristics, genetic algorithms (McConaghy et al.. 2011), and tree-based search (Fan et al., 2021; Zhao & Zhang, 2020). While partially effective, these methods struggle with scalability, efficiency. and adaptability to evolving de- sign requirements. They often require numerous queries and long runtimes to find circuits that meet targets. For example, Fan (Fan et al., 2021) uses tree sampling for au- tomated topology design but faces scalability and practical challenges when handling diverse performance needs, re- quiring over 400 simulation queries per design. Similarly, other search- based methods (Zhao & Zhang, 2020) demand extensive simulations for new specifications. (b) Gener- ative Al-based frameworks, including graph-based based and LLM--ased generative methods. Graph generative models use VAΣs to produce netlists as undirected (Simonovsky & Komodakis, 2018) directed graphs (Dong et al., 2023; Zhang et al., 2019), but lack precise control over compo- nent count, efficiency, or power conversion ratio. Recently, LLMs have been applied to automated circuit topology syn- thesis (Vijayaraghavan et al., 2024; Chang et al., 2024; Lai et al., 2025), leveraging their pattern learning and design generation abilities. Unlike search- based methods, LLMs produce circuits from single prompt after training, en- abling faster generation.
In this work, we present AUTOCIRCUIT-RI (AC-RL), reinforcement learning (RL)-based framework that refines LLM-generated circuit topologies to optimize design objec- tives. Our method employs two training phases: instruction tuning to generate diverse topologies from prompts. fol- lowed by Rl--refinement using Al-based reward models that estimate validity, efficiency, and output voltage. This enables scaling, generalization, and multi-objective opti- mization with minimal manual effort. RL-refinement occurs only during training, not at inference. Empirical results show a ~ 12% improvement in validity and ~~14% gain in efficiency over the best- perffpeerorming LLM baselines, with few-shot generalization beyond 6 components and support for circuits with up 10 components. Key contributions include:
We propose AUTOCIRCUIT- RL, novel RL framework for analog circuit synthesis that targets constraint-driven design.
LLM with RL Refinement:
Superior Performance Evaluation: Our framework's 's eval- uations on 4 and 5-component analog circuits demonstrate superior performance in generating circuits that meet design constraints more effectively than other baseline approaches. Scalability and Generalizability: Using few-shot fine- tuning, our framework generalizes to 6-10 components even with limited data, highlighting its scalability and adaptabil- ity in practical design scenarios.
However, most LLM approaches are limited in scale and flexibility. CircuitSynth ( (Vijayaraghavan et al., 2024) and similar works (Chang et al., 2024) target small circuits (up to six components). AnalogCoder (Lai et al., 2025) generates PySpice code via prompt engineering but depends on fixed synthesis library and lacks iterative refinement, limiting ploration of novel or complex topologies, particularly for power converter circuits requiring high efficiency and spe- cific output voltage constraints. This limitation makes it less extensible for optimizing circuit performance or handling diverse design constraints. Artisan (Chen et al., 2024) is an- other recent effort focusing on operational amplifier design using domain-specifie LLMs. While valuable, it is highly specialized and does not generalize to other circuit families such as power converters. LaMAGIC (Chang et al., 2024) fine-tunes LLMs for netlist generation but omits iterative or performance-driver optimization, restricting adaptability to multi-objective constraints. Auto-SPICE (Bhandari et al., 2024) automates large-seale SPICE netlist generation from textbook schematics (e.g.. Masala-CHAI) but focuses on data creation rather than optimization or refinement. Re- cent advances like AnalogXpert (Zhang et al., 2024) and
2. Problem Statement and Dataset
Given an input instruction, our goal is to produce a netlist with components and their connections. Each entry in the netlist corresponds to node in an undirected graph G, with edges indicating the connections between these nodes as in the netlist. For the choices of encoding the netlist textually, we adopt the "Incident" encoding strategy, recognized for its effectiveness in various graph-related tasks (Fatemi et al., 2024). The center part of Figure 1 illustrates an example of how netlist is encoded. This research investigates dif- ferent model variations that refine Language Models (LMs) to generate the circuit topology netlist. We compare two representation approaches: generating the netlist as lists or employing the text representation with the incident encod- ing method. Through empirical evaluation, our objective is to evaluate the efficacy of these LM variations in accurately and efficiently synthesizing circuit topologies from natural
2

---

AUTOCIRCUIT-RI Reinforcement LaarinnnnnLLLaannnggg Learning-Driven LLM for Automated Circuit Topology Generation
language instructions.
vised learning techniques. This phase focuses on training the model to comprehend instruction prompts that specify the component pool and design constraints, facilitating the efficient generation of valid circuit topologies. To further optimize the circuit topology generation process and ensure compliance with all design constraints, we incorporate rein- forcement learning with Al feedback (RLAIF) (Bai et al., 2022: Lee et al., 2023). The RL refinement phase enhances circuit topology generation by integrating feedback from constraint-specific Al models in three steps: reward model- ing, RL training, and iterative adaptation.
We generated a dataset of switching power converter topolo- gies with 4-10 components (Figure 2(a)). Using Random Search (RS) (Fan et al., 2021), we generated numerous unique netlists. Multiple netlists can represent the same topology by changing component order or node indices, so we report unique designs. The design space for 4- and 5- component circuits is small, allowing near-exhaustive explo- ration; for 6+ components, exhaustive search is impractical. Therefore, we collected 10,000 unique netlists for each com- ponent count from 6 to 10. Each netlist was simulated at 5 duty cycles: 1, 0.3, 0.5, 0.7, and 0.9, yielding 5 times the number of unique netlists as total samples (Figure 2(a)). We used NGSpice (Nenzi & Vogt, 2011) to identify valid cir- cuits and collect output voltage and efficiency values. The circuit includes five external signal ports: Vin, Vout, GND, N-type gate signal, and P-type gate signal (renamed as 'IN', 'OUT", '0', 'GATEN', and "GATEP", respectively). Devices considered are capacitors, inductors, n-type MOSFET (FET- A), and p-type MOSFET (FET-B). Capacitors and inductors have two ports, while MOSFETs have four (drain, gate, source, body). To simplify the design space and accelerate RS generation, FET-A connects gate/body GATEN/C, and FET-B to GATEP/IN. Devices are numbered, with shared ports using one index (Figure 1). Capacitors (10μF), induc- tors (10,H), and MOSFET's use fixed parameters; switching frequency is 200 kHz, input voltage 2V.
3.1. Instruction Tuning
The instruction tuning phase can be considered as a stan- dard supervised finetuning (SFT) step, where instruction prompts specifying the component pool and design con- straints are provided as input to the model, and the corre- sponding circuit topology generations are produced as out- put. We represent the pairs of instruction-cireui topologies as D = {(X,, Y. where X, denotes the instruction prompt and Y, represents the valid circuit topology netlist using the incident encoding method concatenated with the corresponding duty cycle. This phase involves training an autoregressive language model po parameterized by 0 to minimize the negative log-likelihood of the desired circuit topology represented using the incident encoding method. Formally,
For training, we randomly sample approximately 100,000 unique netlists (for 4- and 5-component circuits), each with varying efficiency and output voltage values. The data is cat- egorized into four groups: (a) Group 1 with low efficiency (efficiency < 0.05), (b) Group 2 with moderate efficiency (efficiency between 0.05 and 0.7), (c) Group 3 with high efficiency but minimal output voltage difference from input, and (d) Group 4 with optimal Vout and efficiency, An ex- ample for 4-component circuits is shown in Figure 2(b). We apply weighted sampling strategy in each batch, priori- tizing Group 4 (weight 0.4) and giving the lowest priority to Group 1 (weight 1). This ensures that batches are en- riched with data from Group 4, improving model learning on the most desirable conditions. Instruction prompts are constructed in three categories based on design constraints: (a) Component constraint (only the list of components), (b) Efficiency constraint (components with expected efficiency), and (c) Output Voltage constraint (components with input voltage and expected output voltage).
CSFT = -BDD Σ log To To(
Elog
(I)
where Ta represents the LLM policy and yor denotes all tokens before the 4th token in the circuit topology Y. This objective aims to ensure that the model learns to compre- hend the instruction and generate the circuit topology in the incident encoding method. However, the generated circuit topology may not satisfy all the design constraints related to components, efficiency, validity, and expected output volt- age. To meet such constraints, we utilize Reinforcement Learning with AI Feedback (RLAIF), which learns to re- fine the circuit topology generation process by maximizing rewards associated with specific constraints of interest.
3.2. RL-Refinement with AI Feedback
The RL-refinement phase aims to enhance the circuit topology generation process by leveraging feedback from constraint-specifie Al models. It consists of 3 main steps:
3. Proposed Approach
Our proposed framework, AUTOCIRCUIT-BR R1,, is depicted in Figure 2(c). It consists of two primary phases: instruction uning and RL refinement. In the instruction tuning phase, we fine-tune large language model (LLM) using super-
1.2.1. REWARD MODELING
In this step, our reward model evaluates the appropriate- ness of a generated circuit topology based on the instruction prompt. Canonical reinforcement learning with AI feed-
3

---

AUTOCIRCUIT- RL; Reinforcement CaatiinnnnLLaaann Learning-Driven LLM for Automated Circuit Topology Generation
(a)
0 Comp
Unique
# Unique
# Toal
Valid
(c)
Designs
Notlists
Samplee
Samples
5,031
100,000
500,000
292,811
Iterative adaptation
1,00,000
5,000,000
2,858,400
27,581-28,275
Topology Generation
(b)
Weight Updates
RLAIF Model
x Y)
Circuit Validity
Algorithm: PPO Truiner
Circuit Topology
Circuit Efficiency
0.4
Prompt
Output Voltage
Reference Model
KL Regularization
Prohabilities
-0.5 0.0
0.
1.0 out
1.5
2.0
23
300
Figure 2. ((a) Statisties of Circuit dataset. Vout-Efficiency graph of 4-component circuits the training data. (c) Illustration of the AUTOCIRCUIT-RI framework. Dashed lines depict probability flow, while the dotted line represents iterative adaptation post RLAIF tuning. A prompt guides topology generation, then rewards are calculated with KL-divergence reference model to preserve the original distribution. PPO updates model parameters using rewards, and iterative adaptation further improves the model.
back (RLAIF) trains reward model using labeled prefer- ences in the form of triples (X, Y,,,, Y,), representing the input prompt, preferred topology, and non-preferred topol- ogy, respectively. Recent research suggests that directly using reward scores yields better performance than the tra- ditional RLAIF approach. To achieve this, we employ reward function that evaluates how well the generated cir- cuit topology adheres to different design constraints, such as circuit validity, efficiency, and expected output voltage. To implement this, we train different estimators (felr for all elf E (valid, eff, vout)) to assess circuit validity, effi- ciency, and expected output voltage, and assign correspond- ing scores Self- These scores are then used to compute built
diets output voltage based input parameters, achieving low MSE loss of 8e-3 on the development set.
Using the remaining data in D and the trained estimators, we define a reward function r(X, Y) that assigns a reward to an A..M-generate circuit topology Y as follows:
-1, if svllid 0.6
r(x,
8)
1,
Self or syout meets constraints (2) otherwise
Seff,
reward in the range [], The estimators ing dedicated classification regression models (far), with RoBERTa as the underlying model architecture. Each model is trained on synthetic datasets tailored specifically for each constraint type. The details are as follows:
Invalid topologies receive a negative reward. Valid topolo- gies meeting output voltage efficiency constraints get reward of 1; otherwise, the efficiency estimate is used as the reward to maximize efficiency.
Circuit Validity Estimator binary classifier fvalid is trained on a dataset D comprising valid and invalid circuits, constructed from the aggregated dataset (explained in the Section 2), to determine the validity of circuit topologies. This RoBERTa-based classifier achieves 92% Fi score for binary classification of circuit validity.
3.2.2. RL TUNING
enhance the LLM for generating circuit topologies that better meet the constraints, we employ a reward function r(X, Y) and Proximal Policy Optimization (PPO) (Schul- man et al., 2017). The base model for this refinement is the L.M fine-tuned with the instruction tuning technique discussed in Section 3.1, following the common practice in Reinforcement Learning with Human Feedback (RLHF) (Ouyang et al., 2022). Standard PPO training procedures are then applied optimize the base model using the following reward objective function:
Circuit Efficiency Estimator: A regression model fer estimates circuit efficiency using a subset of the dataset 'D with prompts specifying efficiency requirements. The model achieves an 83% macro Fi score by categorizing predicted efficiency scores into predefined categories.
Output Voltage Estimator: A regression model fvout pre-
CRI -(x, -
-K (TRLAII) (X) (3)

---

AUTOCIRCUIT- RL: Reinforcement Learning-Driver LLM for Automated Circuit Topology Generation
Here, KL represents the Kullback-Leibler divergence, and 1) is hyperparameter controlling the penalty for divergence. This penalty helps prevent the model from getting trapped in local optima or straying too far from the original distribution of the supervised instruction-tunel model.
specific LLM. In contrast, AUTOCIRCUIT-RL is currently trained using power converter designs and combines in- struction tuning with RL refinement to handle diverse user prompts and optimization goals. Although these prior works are not directly applicable to our constraint-driven setting, future adaptations could make comparisons more feasible.
3.2.3. LTERATIVE ADAPTATION (IA)
Given these considerations, we evaluate AUTOCIRCUIT- RL against the following baselines: Zero-Shot Genera- tion: Prompts with component pools and design constraints are directly fed into large language models (LLMs) like Llama-2 (13b) and FLan-UL2 (20b) without fine-tuning, aiming to generate circuit topologies; In-Context Learn- ing (ICL): This approach uses circuit generation demon- strations, combining input prompts with component pools, design constraints, and corresponding output circuits within the prompts. It leverages the in in-context learning ability of LLMSs such as Llama-2 (13b) and Flan-UL. (20b), explor- ing different numbers of examples (j E {5, 10, 20)) and experimenting with incident encoding and netlist structures; Prompt Tuning: This method fine-tunes LLMs like Llama- (13b) and Flan-U1.2 (20b) for eircuit topology generation using a Prompt-tuned Model (Lester et al., 2021), which learns task-specific soft prompts while keeping model pa- rameters unchanged. We test with 100 trainable soft prompt tokens (p100); Vanilla Fine-Tuning: Standard fine-tuming is conducted on the LMs listed above. The primary objective is to minimize the negative log-likelihood for generating circuit topologies; Gumbel-Max Fine- Tine-Tnnig: Drawing ideas from a prior study (Vijayaraghavan et al., 2024), we integrate multiple objectives optimizing for circuit validity and efficiency using the Gumbel-Max trick. This approach refines models fine-tuned using Llama-3 (8b) and MPT-7b architectures, allowing for more efficient circuit generation while maintaining structural validity constraints; GRAPH- VAE: This method models circuit netlists as undirected graphs and uses a variational autoencoder (VAE) to gen- erate graphs from continuous embeddings (Simonovsky & Komodakis, 2018). We did not implement DAG-based meth- ods(Dong et al., 2023; Zhang et al., 2019), because notice- able subset of circuits in our datasets are not DAGs. For cir- cuits with 6-10 components, the generation process is con- ditioned on a SentenceBERR--encodde label vector (Reimers, 2019) derived from the input prompt, enabling controlled and guided sampling during inference; AUTOCIRCUI-RLL We introduce AUTOCIRCUIT-RL. work designed to enhance the circuit topology generation process using RL and iterative adaptation.
Additionally, we explore the concept of iterative adaptation as means refine the circuit topology generation process further. Leveraging the circuits generated in the previous steps, we aim to enhance the overall synthesis task by it- eratively adapting the model based on the sampled valid and highly efficient synthesized circuit topologies. Utiliz- ing the nucleus sampling approach, we specifically target circuit topologies with an efficiency score (sert) exceed- ing 0.7. This criterion ensures that the selected eircuits not only satisfy validity constraints but also exhibit high operational efficiency. To initiate the iterative adaptation process, we synthesize dataset comprising 10,000 such samples. Starting from the RI.-tuned model described in Section 3.2.2, we iteratively refine the model using Equa- tion 3, incorporating insights from high-quality topologies into the RL-refined model from the previous iteration. Upon completion, the final model is used directly for inference, with no further tuning. In our experiments, we evaluate this iterative adaptation's 's effectiveness to enhance the circuit topology generation process.
4. Experiments and Results
In this study, we utilized the following Language Mod- els (LMs) for generating circuit topologies: GPT-Neo- 2.7 (Black et al., 2021), Stablel.M-3BB44111 Llama-3-8b (Grattafiori et al., 2024), and MPT-7b (Team et al., 2023). More details on the baseline models and implementation are provided in the Appendix A, and B.
4.1. Baselines
Our primary focus is on leveraging LLM-based genera- tive methods for circuit design synthesis and benchmarking their potential. We compare different L..M-based meth- ods as baselines and include Graph (Simonovsky & Komodakis, 2018), non-LLM baseline, to highlight the superiority of our method in both efficiency and perfor- mance. While recent LLM- -baeed frameworks like Analog- Coder (Lai et al., 2025) and Artisan (Chen et al., 2024) provide valuable contributions, their methodologies and ap- plication domains differ from ours. AnalogCoder employs a training-free free prompt strategy with a fixed synthesis library, focusing on general analog circuits, and does not specifically address power converters, which require nuanced handling of constraints such as efficiency and output voltage. Artisan is tailored for operational amplifier design using domain-
comprehensive frame-
4.2. Metrics
In our evaluation setup, we report different metrics based on sampling 500 unique circuit topologies from each of the trained models. (a) Circuit Validity Score represents the fraction of unique circuit topologies estimated as valid,
5

---

AUTOCIRCUIT- RL: Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation
Table 1. Evaluation results for all methods on 4-component (4C) and 5-component (5C) circuits. Cireuit validity and efficiency are measured using both classifier (column 2 and 4) and simulator (column 3 and 5). The DGR and success rate a for different categories of constraints: component (C), efficiency (C+B), output voltage (C+V) and overall (O) are also reported.
Models
Evina ())
E(Is.ana (w))
E(Jo (7))
Elss (7))
DGR p
Success Rate (%)
Prompt Tuning
4C
5C
4C
5C
4C
5C
4C
5C
C
C+B
C+V
0
Alama-2-132 p-00
54.60
50.20
54.50
53.40
53.60
43.62
52.45
47.97
5.39
80.94
33.89
34.02
43.35
57.40
51.60
57.35
56.00
56.20
44.50
56.08
49.64
4.76
84.09
36.54
34.67
45,30
Vanilla & Gumbel-
Multi-Objecnti Fine Tuning
GPT-NeOFT
60.50
57.80
60.60
57.78
57.90
54.70
56.73
54.49
2.98
89.52
40.28
37.62
49.06
StableLMPr
59.42
58.10
60.20
59.00
58.20
55.0
59.51
53.27
2.67
89.04
39.76
37.96
48.89
Llama-3rr
66.78
63.50
66.80
63.80
62.8
61.3
61.15
62.97
2.10
95.76
68.04
58.06
69.79
MPT-7bpr
64.96
61.65
65.00
60.50
60.50
59.2
60.23
62.23
2.26
95.32
67.85
58.28
69.52
Gumbelatama-3
67.60
65.75
67.32
64.19
67.15
64.38
63.87
63.15
2.19
96.04
69.56
60.36
71.27
GumbelMaP--7
67.16
63.50
66.42
63.48
66.30
64.42
63.19
63.64
2.32
95.80
68.30
60.22
70.68
AUTOCIRCUIT
RL (our approach)
AC-RLriamn-3
75.11
73.46
74.48
73.96
74.20
73.60
71.65
72.22
1.29
99.08
80.90
71.30
80.69
w/o IA
71.08
68.32
72.78
68.96
71.60
69.70
69.50
68.68
1,46
98.26
76.75
70.10
78.39
AC-RLAPT-T7
74.20
72.65
74.32
72.34
73.40
72.70
72.08
71.94
1.34
99.05
79.85
71.64
80.41
w/o IA
70.88
69.56
71.06
69.23
70.10
69.50
67.37
69.08
1.53
98.03
76.52
68.50
77.61
denoted as E(Jar(j)). Here, elf {valid, Svatia) refers to the validity estimated by the classifier or the simula- tor, respectively. We consider a circuit valid if its validity score from the classifier exceeds 0.6; (b) Circuit Efficiency Score refers to average efficiency of the generated circuits using our efficiency regressor or the NGSpice simulator, denoted as E(fer (1)) and E(fs... (ū)), respectively. The NGSpice simulator evaluates the validity and efficiency of given netlist by verifying its electrical characteristics and performance metrics through detailed circuit simulations. It evaluates parameters such as voltage levels, timing, and power consumption of the circuit topology with certain duty cycles under given conditions; (c) Duplicate Generation Rate (DGR), denoted by p. indicates the number of circuit topologies required to be sampled from the model to obtain unique cireuit topology design. Formally, p is calculated as the number of topologies generated divided by the num- ber of unique topologies (500 in our case); and (d) Success Rate computes the percentage of valid circuit topologies that successfully meet certain design constraints for each category of prompts (as in Section 2): Component constraint (C), Efficiency constraint (C+E), Output Voltage constraint (C+V) and overall success rate (O). Figure/ presents sample success scenario with efficiency constraint.
(a) circuit efficiency and (b) success ratio, defined as the proportion of generated circuits satisfying functional and design constraints. Figure 3 illustrates these convergence curves for circuits composed of 4 and 5 components, trained using the Llama-3 backbone over ~ 25, 000 training steps with a batch size of 16. The training exhibits several char- acteristic phases. In the initial phase, both efficiency and success ratio increase gradually with noticeable oscillations, reflecting the model's early adaptation to reward signals. This is followed by a phase of rapid improvement, where the RL agent learns effective circuit design strategies. In- termediate fluctuations arise as the model explores diverse topologies while balancing validity, constraint satisfaction, and efficiency. Eventually, the curves stabilize and plateau, indicating convergence a policy that consistently gener- ates valid, high-quality circuit topologies. These results show AUTOCIRCUIT-RL steadily improves design capabili- ties and sustains robust performance as circuit complexity grows, with only minor efficiency degradation observed from 4- to 5-component circuits.
4.4. Results Overview
The evaluation results, summarized in Table 1, show that AUTOCIRCUIT outperforms all baselines across vari- ous metrics for circuit topology synthesis. Notably, smaller language models tuned with our method outperform larger prompt tuning-bass based models, generating unique designs faster with higher success rate in meeting design con- straints. Our LLMd-based approach is efficient, requiring
4.3. RL Convergence Analysis
We evaluate the learning dynamics of the proposed AUTOCIRCUIT-RL framework by analyzing the conver- gence behavior of two key metrics during RL. refinement:
6

---

AUTOCIRCUIT- RL: Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation
RL Convergence Curve: Efficiency (6Cvs
model lacks fine-grained control.
Our analysis reveals that validity failures predominantly arise from minor inconsistencies in node assignments rather than fundamental topological errors, and such failures can typically be resolved with minimal structural modifications. Regarding efficiency constraint errors, most of the circuits that fail to meet the specified thresholds do so by a rela- tively small margin, suggesting that the model generally approximates the desired performance metrics. These de- viations often reflect inherent trade-offs with other design parameters such as output voltage and duty cycle. A limited number of outlier cases exhibit larger deviations from the efficiency targets, typically caused by complex interactions between circuit components. Collectively, these findings indicate that while the model internalizes key principles design,
14-
RL Convergence Curve: Success Ratio
SacAs tallo
66-
5009
2500
Training
of both structural integrity and performance-aven there remains scope for further improvements in constraint calibration and optimization. Detailed examples and further discussion can be found in Appendix C.
Figure 3. RL convergence curves for AUTOCIRCUIT-LL with Llama-3 over 255,000 training steps. Top: Efficiency for 4- and 5-component circuits; Bottom: Success ratio of valid and constraint-satiifyingg topologies.
4.6. Effectiveness of AUTOCIRCUIT-RI
2.7 seconds per design generation once trained (refer Section D for runtime analysis). In contrast, traditional search-based algorithms (Fan et al., 2021) are computation- ally expensive and time-consuming, often taking hundreds of seconds to converge on a target design.
Impact of Prompt Tuning Experiments with prompt tuning of Flan- Maan-UL//Llama- models (approximately 20b param- eters) with a restricted dataset, facilitated the production of netlist-like structures. This represents a substantial en- hancement compared the performance of the same mod- els under zero-shot in-context learning (ICL) conditions. Nevertheless, these fine-tuned models lag behind smaller language models (such as GPT.NNeo/StableL.MM fine-tuned using our method in terms of both efficiency and validity of the generated circuits. Additionally, our models demon- strate a higher success rate in meeting constraints compared to the larger prompt-tuned tune language models. Despite the ad- vantages of fine-tuning these larger models, we emphasize that fine-tuned models with lower capacity can still achieve effective performance relative to larger prompt tuning-based models for circuit topology synthesis.
Usability of Zero-Shot and ICL Methods We observe notable disparity in performance between zero-shot gener- ation techniques and fine-tuned methodologies. Zero-shot generation struggles produce valid netlist-like structures essential for subsequent classification or simulation tasks. Despite using in-context learning (ICL., where the model is exposed to sample prompts and corresponding circuit generations, the improvement in generating comprehensive netlist-like structures is minimal. Additionally, increasing the number of in-context examples j yielded diminishing returns. As result, these incomplete structures cannot be used for accurate metric computation, so we exclude the zero-shot and ICL results in Table 1.
Comparison with Vanilla Fine-tuning When contrasting our methodology, which entails iterative refinement via re- inforcement learning, with the basic fine-tuning of different architectures, we note significant enhancement in the effec- tiveness of our approach. Our methodology, which entails it- erative refinement via reinforcement learning, demonstrates significant improvement in effectiveness compared to ba- sic fine-tuning of GPT-Neo and StableLM architectures. This improvement is evident in generating valid circuits that meet design constraints, underscoring the efficacy of our approach in synthesizing valid topologies and accelerating the discovery of unique designs.
4.5. Error Analysis
We conduct a qualitative error analysis of the AUTOCIRCUI-RLL framework by evaluating the model- generated circuits through post-hoc SPICE simulations. The analysis focuses on two primary failure modes: validity errors, where circuits fail to simulate due to structural violations such as incorrect node assignments or connectivity issues, and efficiency constraint errors, where the generated circuits do not meet the efficiency thresholds specified in the generation prompts. This systematic evaluation enables the identification of recurring failure patterns and provides insight into the model's behavior near constraint boundaries, highlighting specific areas where the
Effect of Using Gumbel-Max Trick for Multi-Objective Optimization Building on a prior study (Vijayaraghavan et al., 2024), we optimize circuit validity and efficiency

---

AUTOCIRCUIT-RIE RL: Reinforcement Learning-Driver LLM for Automated Circuit Topology Generation
synthesis.
Component Pool Adherence (C) Ensuring adherence to the component pool constraint is pivotal for practical appli- cability, achievable with minimal tuning. Generally, models fine-tuned or optimized with reward models exhibit superior adherence to the specified component pool. For instance, prompt tuning-basse based models, even with minimal tuning, achieve moderate success rates, with a values ranging from 81% 84%. Remarkably, models trained using complete AUTOCIRCUIT-RI approach showcase the high- est success rates, with a values surpassing ~ 98%.
Valiiity
Emfciienyy
Figure 4. Few-shot fine-tuning results for 6-10 components, with models tuned using k- e-examples of varying circuit topologies.
Table Overall success rate of GRAPHVAE. Prompt Tuning and AUTOCIRCUIT-RI Rr. for 6-10 component circuits with k 1000.
Models
6c
7C
8C
9C
10C
50
GRAPHVAE
21.5
20.6
16.1
15.9
12.8
PROMPT TUNING
39.7
36.3
35.9
33.2
32.1
AC-RL
65.5
63.8
63.2
60.4
58.5
using the Gumbel-Max trick, yielding notable gains over standard fine-tuned models. However, its performance still lags behind AUTOCIRCUITRLL by ~ 9% in circuit validity and efficiency. A key limitation of the Gumbel-Max trick its inability to adaptively optimize multiple objectives. Unlike reinforcement learning (RL), which refines strate- gies through iterative Al feedback, Gumbel-based methods don't t adjust based prior evaluations and lack mech- anism to balance competing objectives, often leading to suboptimal designs. In contrast, our RL approach navigates these trade-offs effectively, explores a broader design space, and avoids premature convergence, demonstrating superior performance in circuit validity and efficiency.
of Componeuts
Figure 5. Plot SuccessRate@m of AC-RL for m E (1,3,5).
Efficiency (E) & Output Voltage (V) Adherence Meeting
efficiency and expected output voltage constraints chal- lenging in comparison to the component pool constraint, par- ticularly when expected values are specified in the prompts. However, models optimized with reward models show no- table improvements in meeting constraints. Particularly, models trained using AUTOCIRCUIT-RR RL exhibit superior performance in satisfying efficiency and expected output voltage requirements compared to their counterparts.
Effect of Iterative Adaptation To assess the significance of the iterative adaptation strategy for task, we conducted an experiment where we sampled circuit topology genera- tions from a model trained using reward models but without incorporating the iterative adaptation strategy. Our results reveal that the iterative refinement enabled by iterative adap- tation substantially improves the overall model performance across various metrics. Notably, the lack of an iterative adaptation strategy leads to a more pronounced decline in performance, particularly notable in the circuit efficiency score, where we observed a 8% decrease.
4.8. Generalization to Complex Scenarios
To assess the generalization capability of our framework, we evaluated it on circuits with 6-10 components. Following the procedure in Section 2, we constructed limited dataset and applied few-shot fine-tuning with k {250, 500, 1000} examples. The base model for fine-tuning was the Ac-RI. model previously trained 4- and 5-component circuits (see Table 1). We adopted a sampling strategy similar to that used for 4/5-component circuits, as shown in Figure 2(b). We compared our nop-performing AUTOCIRCUIT-RI (Llama-3) approach against GRAPHIVAE and prompt-tuned Flan-ul-20b, focusing on circuit validity and efficiency, mea- sured by the NGSpice simulator. As shown in Figure AUTOCIRCUIT-RO RL showed consistent improvements with increasing k. achieving validity and efficiency scores near- ing 50% with fewer than 500 examples. Table 2 summarizes the success rates for all methods with & = 1000, highlight- ing the superior performance of AUTOCIRCUIT-R RL. The higher success rate underscores the effectiveness of reward
4.7. A Adherence to Design Constraints
This section examines how different models meet three key design constraints for 4- and 5-component circuits. We eval- uate success rate (a) for component usage (C), efficiency (C+E), and output voltage (C+V), using a 20-40-40 prompt split per constraint. AUTOCIRCUIT-RE RL excels across con- straints, demonstrating RL's effectiveness in eircuit topology
8

---

AUTOCIRCUIT RL: Reinforcement Learning-Driver LLM for Automated Circuit Topology Generation
models in optimizing efficiency and validity. These findings show AUTOCIRCUIT-RI, potential to scale to higher com- ponent circuits using minimal fine-tuning, though address- ing more complex design constraints for different circuit types remains a future research area.
alization. This ability to perform well with limited data distinguishes it from conventional methods relying on ex- tensive labeled datasets.
potential
Generalization to Increasing Circuit Complexity AC-RL scales effectively to circuits with 6-10 components, out- performing baselines like GRAPHVAE and prompt-tuned FLAN-UL-20B. While complexity increases, reducing suc- cess rates, more training data significantly enhances perfor- mance. Additionally, SuccessRate @ analysis shows that generating multiple circuits per prompt improves conver- gence with minimal added computation.
Analysis of Success Rates& & Sampling Strategies Com- paring success rates in Tables 1 and 2, we observe per- formance decline in AC-RL as circuit complexity increases. However, this drop is due to differences in training and evalu- ation setups, as the model was extensively trained on 4C and SC circuits, whereas 6C and beyond used few-shot setup with only 1,000 samples. Despite this, the model achieves >60% success with minimal training data, demonstrating good generalization. As shown in Figure 4, increasing train- ing data improves performance, reinforeing the model's ability to learn from limited data. We also analyzed Suc- cessRate @ m, where multiple generations per prompt (m) increase the likelihood of success. As seen in Figure 5, Suc- cessRate @3 3 and SuccessRate@$ consistently outperform SuccessRate @ I. showing that additional sampling improves results. The average generation times for m = and m are 7 and 9 seconds, respectively, achieving higher suc- cess rates with minimal added computation and remaining significantly faster than traditional Al-based search methods, which require hundreds of seconds.
Extending AC-RL to diverse circuit architectures and inte- grating RL with advanced sampling could further optimize high-dimensional design spaces. Addressing constraints like power consumption and component parameter estima- tion via adaptive reward modeling remains promising avenue. Overall, AUTOCIRCUIT-RI offers a transforma- tive approach to circuit topology synthesis, excelling in efficiency, constraint adherence, and generalization with minimal data. Further enhancements could solidify its role as a cornertonn in AI-driven electronic design automation.
6. Conclusion
In this work, we introduced AUTOCIRCUIT-RU (Ac-RL), a framework for automating analog circuit topology synthesis via a two-phase training process. The first phase employs instruction tuning, where an LLM generates initial topolo- gies from structured prompts encoding design constraints like component pools and target efficiency or output voltage, ensuring basic feasibility. The second phase, RL refine- ment, updates the model using Al-based reward feedback to improve validity, efficiency, and output voltage. This refine- ment is applied only during training to improve generation quality, enabling faster inference. Empirical results show that AC-RL outperforms prior methods, generating ~ 12% more valid circuits and improving efficiency by ~ 14% for 4- and 5-component circuits. It demonstrates strong gen- eralization, achieving near 50% validity and efficiency for larger circuits with minimal training data. Additionally, it reduces duplicate generation by ~ 38% compared to other baselines and, with only 1,000 training examples, generates designs where >60% meet the specified constraints. Ac- RL's reward-driven optimization adapts to evolving design constraints, setting a new benchmark for Al-driven design automation. Our findings aim to accelerate the develop- ment of efficient, adaptable methodologies in analog circuit automation, enabling faster, reliable synthesis for circuit topologies with reduced training data needs. Future work will extend AC-RL to support more complex designs, incor- porate advanced sampling, and refine power models, further advancing electronic design automation.
5. Discussion
Our results demonstrate the effectiveness of proposed Ac- RI. in advancing circuit topology synthesis by iteratively refining circuit generation using reinforcement learning. Its key advantages are as follows:
RL for Improved Validity and Efficiency Unlike tradi- tional methods, AC-RL adapts circuit generation using re- ward feedback. This iterative process yields higher validity and efficiency, outperforming zero-shot and ICL baselines. Its ability optimize multiple objectives while adhering to constraints highlights its robustness.
Accelerated Discovery of Novel Circuits AC-RI. reduces duplicate generation rates (DGR) by ~ 38% compared to fine-tuning methods, enabling faster convergence to unique and effective circuit topologies. This accelerates design space exploration while minimizing redundant computa- tions, making it highly effective for circuit automation.
Enhanced Constraint Adherence With up to 80% success rates in constraint adherence, AC-RI. efficiently balances efficiency and output voltage while ensuring feasibility. Un- like heuristic-based methods that require extensive com- putational resources, its reward-driven approach adapts to complex constraints with significantly better performance. Scalability with Limited Training Data AC-RL achieves over >60% success rates generating valid circuits with only 1, 000 training examples, demonstrating strong gener-
9

---

AUTOCIRCUIT-RL.
Leinoreemeennnnnnnnnnnn Learning-Driven LLM for Automated Circuit Topology Generation
Impact Statement
via automated spice netlist extraction from analog circuit diagrams. arXiv preprint arXiv:2411.14299, 2024.
The proposed AUTOCIRCUIT-RL framework significantly advances the field of analog circuit topology synthesis by in- tegrating large language models (LLMs) with reinforcement learning (RL) for constrained topology generation. Our ap- proach addresses key challenges in circuit synthesis, such as scalability, efficiency, and adaptability to diverse design specifications, which are critical in modern electronic de- sign automation (EDA). By overcoming the limitations of traditional search-based and generative methods, our frame- work enables efficient circuit generation while reducing the number of simulation queries. This not only enhances the practicality of AI-driven circuit design but also provides scalable solution capable of generalizing to circuits with 6 to 10 components using few-shot fine-tuning. Our results demonstrate superior performance in generating circuits that meet design constraints more effectively than existing approaches, paving the way for future research in topology- aware AI design methods.
Black, S., Leo, G., Wang, P., Leahy, C.,, and Biderman, S. GPT-Neo: Large Scale Autoregressive Language Model- ing with -ees--Tensrr,low, March 2021. URL https: / //doi.org//// / 10. 5281 wwtp//eeooooooooo//// 02977t555297777 5297715. 5tt.0..............
Chang, C.-C., Shen, Y., Fan, S., Li, Zhang, S., Cao, N., Chen, Y., and Zhang, X. Lamagic: Language-modke based topology generation for analog integrated circuits. In the 41st International Conference on Machine Learn- ing (ICML), pp. 1-8, 2024.
Chen, Z., Huang, J., Liu, Y., Yang, F., Shang. L.,, Zhou, D., and Zeng, X. Artisan: Automated operational amplifier design via domain specific large language model. In Proceedings of the 61st ACM/IEEE Design Automation Conference, pp. 1-6, 2024.
Dong, Z., Cao, W., Zhang, M., Tao, D., Chen, Y., and Zhang, X. Cktgnn: Circuit graph neural network for electronic design automation. In International Conference on Learn- ing Representations (ICLR), pp. 1-20, 2023.
Beyond technical improvements, our work has broader so- cietal and ethical implications. By automating complex as- pects of circuit design, AUTOCTRCUIT-RR RL. reduces reliance on extensive human expertise, potentially lowering the bar- rier to entry for new designers and expanding accessibility to hardware design in low-resource environments. This de- mocratization of circuit synthesis could foster innovation and accelerate technological advancements across indus- tries. However, it is crucial to ensure that AI-driven design tools remain transparent and do not introduce unintended biases in circuit topology selection, which could lead to over-reliance on specific architectures. Additionally, given the increasing energy demands of AI-driven automation, future work should consider optimizing the computational efficiency of training and inference phases to minimize vironmental impact. By addressing these aspects, our work contributes to responsible AI adoption in EDA, ensuring that automation enhances creativity and inclusivity in circuit design rather than reinforcing existing barriers.
Fan, S., Cao, N., Zhang, S., Li, J., Guo, X., and Zhang, X. From specification to topology: Automatic power con- verter design via reinforcement learning. IEEE/ACM International Conference On Computer Aided Design (ICCAD), pp. 1-9, 2021.
Fatemi, B., Halcrow, J., and Perozzi, B. Talk like a graph: Encoding graphs for large language models. In The Twelfth International Conference on Learning Represen- tations, 2024. URL https: /pwp///wppeeeeeeeeeeeepeee nett/ I
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. The llama 3 herd of models. arXiv preprint 2007. 2024.
Hakhamaneshi, K., Werblun, N., Abbeel, P., and Stojanović. V. Bagnet: Berkeley analog generator with layout opti- mizer boosted with deep neural networks. In JEEE/ACM International Conference on Computer-Aided Design (IC- CAD). pp. 1-8, 2019.
References
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKin- non, C., al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint 22200:221..08073, 2022.
Jang, E., Gu, S., and Poole, B. Categorical repa- rameterization with gumbel-softme arXiv preprint 1X110111 2016.
Bengio, Y., Léonard, N., and Courville, A. Estimating or propagating gradients through stochastic neurons for con- ditional computation. arXiv preprint arXiv:1308.3433 2013.
Lai, Y., Lee, S., Chen, G., Poddar, S., Hu, M., Pan, D. Z., and Luo, P. Analogcoder: Analog circuit design via trannin-frre code generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 379 387, 2025.
Bhandari, J., Bhat, V., He. Y., Garg, S., Rahmani, H., and Karri, R. Auto-spice: Leveraging llms for dataset creation
10

---

AUTOCIRCUIT RL: Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation
Lee, H., Phatale, S., Mansoor, H., Lu, K., Mesnard, T., Bishop, C., Carbune, V., and Rastogi, A. Rlaif: Seal- ing reinforcement learning from human feedback with ai feedback. arXiv preprint wwxi:2230.000267, 2023.
Wang, H., Wang, K., Yang, J., Shen, L., Sun, N., Lee, H., and Han, S. Gen-rl circuit designer: Transferable transis- tor sizing with graph neural networks and reinforcement learning, In ACM/IEE Design Automation Conference (DAC), pp. 1-6, 2020.
Lester, B., Al-Rfou, R., and Constant, N. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.0869, 2021.
Zhang, H., Sun, S., Lin, Y., Wang, R., and Bian, J. Analogx- pert: Automating analog topology synthesis by incorpo- rating circuit design expertise into large language models. arXiv preprint 2rx2 19824, 2024.
Loshchilov, I. and Hutter, F. Decoupled weight decay regu- larization. arXiv preprint arXiv:1711.05101 2017.
Zhang, M., Jiang, S., Cui. Z., Garnett, R., and Chen, Y. D- vac: A variational autoencoder for directed acyclic graphs. Advances in neural information processing systems, 32, 2019.
McConaghy, T., Palmers, P., Steyaert, M., and Gielen, G. Trustworthy Genetic Programming-Basse Baaed Synthesis of Analog Circuit Topologies Using Hierarchical Domain- Specific Building Blocks. IEEE Transactions on Evolu- tionary Computation, 15(4):557-570, 2011. ISSN 1941- 0026.
Zhao, Z. and Zhang, L. An Automated Topology Synthesis Framework for Analog Integrated Circuits. IEEE Trans- actions Computer-Aided Design of Integrated Circuits and Systems, 39(12):335-4333 2020. ISSN 1937-4151.
Nenzi, P. and Vogt, H. Ngspice users manual version 23. Experiments/ ngspice23-manual pdf. 2011.
Ouyang. L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advunces in neural information processing systems, 35:27730-27744, 2022.
Reimers, N. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908. 10084, 2019.
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint 2007.1707.06633 2017.
Shen, J., Chen, Z., Zhuang. J., Huang, J., Yang, F., Shang, L., Bi, Z., Yan, C., Zhou, D., and Zeng, X. Atelier: An automated analog circuit design framework via multiple large language model-based agents. Authorea Preprints, 2024.
Simonovsky, M. and Komodakis, N. Graphvae: Towards generation of small graphs using variational autoencoders. In Artificial Neural Networks and Machine Learning- ICANN 2018: 27th International Conference Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part 1 27, pp. 412-422. Springer, 2018.
Team, M. N. et al. Introducing mpt-7b: A new standard for open-source, commercially usable 1lms, 2023.
Vijayaraghavan, P., Shi, L.. Degan, E., and Zhang, X. Cir- cuitsynth: Leveraging large language models for circuit topology synthesis. 2024 IEEE LLM Aided Design Workshop (LAD), pp. 1-6. IEEE, 2024.

---

AUTOCIRCUIT RL; Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation
A. Appendix: Baseline Model Details
compare AUTOCIRCUIT-RL against various baseline models, including both LLM-based and non-LLM approaches, to assess its efficiency and performance in circuit topology synthesis.
GPT-Nee-2.77 (Black et al., 2021) 1 A 2.7B parameter transformer- bassed model developed by EleutherAI, following GPT-3's architecture. It was trained on 420B tokens over 400,000 steps using masked autoregressive modeling with cross-entropy loss.
StableLM-3B-4EI1 :: A 3B parameter decoder-only model pre-trained on IT tokens over 4 epochs from diverse English and code datasets. Referred to StableLM in our work.
Llama-3-8B' 9; An 8B parameter model from the Meta Llama 3 family, incorporating supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to enhance alignment with human preferences.
MPT-7B 4: A 7B parameter decoder-style transformer trained on IT tokens of English text and code. It employs modified transformer architecture optimized by MosaicML for efficient training and inference.
CIRCUITSYNTH-GUMME A variant of (Vijayaraghavan et al., 2024) incorporating:
1. Training circuit validity and efficiency classifier to estimate the probability of generated eircuit being valid.
2. Fine-tuning LLM to generate circuit topologies.
3. Refining outputs using the classifier while enforcing circuit validity and efficiency constraints.
The training objective combines standard negative log-likelihood loss (LLLM) with circuit validity and efficiency loss. Since LLMs operate in discrete spaces, we employ Gumbel-softmax (Jang et al., 2016) for continuous relaxation, enabling gradient-baae -ased optimization.
GraphVAE (Simonovsky & Komodakis, 2018): A non-LLM baseline for circuit generation. The encoder consists of two graph convolutional layers (32 and 64 channels) with identity connections, batch normalization, and ReLU activation, followed by a fully connected (FC) layer producing a 256-dimensional latent representation. The decoder has three FC layers (256, 512, 1024 channels) with batch normalization and ReLU, followed by a parallel triplet FC layers output graph tensors. Training was performed for 50-75 epochs using Adam (learning rate le -3, 3,31 = 0.5). A sentence transformer (Reimers, 2019) processes natural language prompts, followed by FC layer align its representation with the encoder output size.
Prompt Type
Prompt
Sample Circuit
[['FET-A-2", '5', '0').
Generate 4-companent circuit with n-type MOSFETs: FET-A-0, FET-A-1 A-1Lam and FET-A-2; and p-type MOSFET: FET-B-0; representing different nodes:
['FET-B-0', "OUT].
Component Constraint
['FET-A-I', 00, 'NN].
('FET-A-2' FEE--B-' /ET-A--', "FET-A-0']
['EET-A-,", "5. "OUT"I
Generale c-mmpnnent circuit with capacitons: capacitue O and capacitor I; type MOSFET: FET-A-0; and p-type MOSPET: FFT-B-0 representing different nadec: L'capacitor-1 I', FEET-A-O" 'capecioo-OO, "FET-B-O'] with Vout less than 1.5V when Vin equals 2V
I''sppccion I", IN. '10'),
[FET-A-O", 'OUT, IN').
Output Vollage Construint
('sapacitor-O". 10).
(HEET---O,, "OUT, [10]]
['inducton--', ",, 'OTT).
Generate a 4-compouent circuit with inductor: inductor-O, n-type MOSFET: FET-A-0, p-type
I'EE--A-O, 'OUT. '6).
Elficiency Construint
FET-B-O: capacitor: capacitor-O; representing different nodes: ('inductor 0', "FET O', B.T.B.".. 'capacitor 0') with efficiency greater than 0.6
FEET--O- "N",, "6,.
O' "OLT", o']]
Table 3. Sample prompts of each prompt type and their corresponding sample circuit topologies.
hnttps:
3b- 4elt
httpppp////ngiiiraaeece////tttt/ ama-
12

---

AUTOCIRCUIT- RL: Reinforcement Learning-Driven LeaanieinnnnLLLiini-inai LLM for Automated Circuit Topology Generation
B. Appendix: Implementation Details
provide the implementation details of our experiments conducted with the official PyTorch v2.2.0 release binary package, compiled with CUDA 11.8, utilizing NVIDIA V100 GPUs with 32 GB of memory.
Two-Phase Training: This setup applies to both instruction tuning and RL-refinement phases. We plot the training data on a Vout-Efficiency graph (an example for 5-component circuits is shown Figure 6) and categorize the data into four main groups: (a) Group with low efficiency (efficiency lower than 0.05), (b) Group 2 with moderate efficiency (efficiency between 0.05 and 0.7), (c) Group 3 with high efficiency but the output voltage shows minimal difference from input voltage, and (d) Group 4 with optimal Vout and efficiency. To optimize the training process, apply weighted sampling strategy in each batch, giving the highest priority to Group 4, which represents the optimal conditions, and the lowest priority to Group I, which has the least efficiency. The sampling weights are allocated as follows: 0.1 for Group 1, 0.25 for Group 2, 0.25 for Group 3, and 0.4 for Group 4. This approach ensures that batches are enriched with data from Group 4, allowing model to better learn from the most critical and desirable conditions.
Training conducted over 4-6 epochs using shuffled data from the training split, with model checkpoints saved based on the best performance on the validation split. To manage memory efficiently, we employ gradient checkpointing. We use the AdamW optimizer (Loshchilov & Hutter, 2017), setting beta parameters to 0.9 and 0.95, and epsilon value of 1.0e-8. The learning rate is set to 0.95e-5, and the seed is fixed at 42 to ensure reproducibility. During training, we assess performance by evaluating subset of 100 sample generations, using consistent evaluation settings. If the performance in the current epoch exceeds that of the previous one, we save the checkpoint.
E-0.5V+0.01
1.0
0.8
E.
Effiiincy
group
group
group
0.0
group
E=0.05
V out
Figure 6. Vfut-Efficiency graph of 5-component circuits in the training data.
Iterative Adaptation: The iterative adaptation, explained in Section 3.2.3, is performed over 3-5 iterations depending the base model used. Each iteration involves 2-4 epochs of RL refinement and uses a dataset of 10,000 high-quality circuit samples obtained via nucleus sampling. This process improves performance progressively with each stage. The final adapted model is directly used for inference without further tuning.
We assess all models by generating 1,000 unique sample circuit topologies from each, using a combination of nucleus sampling and top-k sampling techniques. These generated topologies are then analyzed using validity, efffccincy, and output voltage estimators identify the designs that both valid and efficient, meeting the specified design criteria.
Inference:
C. Error Analysis
C.1. Validity Errors
We begin our enror analysis by examining one common failure mode encountered in generated circuit topologies: validity errors, as identified by SPICE simulation. A netlist is considered invalid it violates essential electrical constraints such as correct node referencing, grounding, or connectivity, which causes simulation failures.
13

---

AUTOCIRCUIT RL; Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation
Despite the strong overall performance of model, small subset of generated netlists exhibit such validity issues. These typically stem from improper inconsistent node assignments rather than errors in component selection overall topology. A representative example of an invalid netlist generated by the model is:
[L'FET-B-1', 'IN' ' ['FET-A-O', ro', , 'N' ['FET-B-O', YOUT', '7'], ('inductor-0', '', 1
This netlist fails validation primarily due to ambiguous or incorrect node labeling, which disrupts the circuit's connectivity and prevents successful simulation. However, with minimal adjustments, specifically refining the node assignments to properly reference outputs and grounds, the netlist can be made valid:
[['FET-B-1', 'I'' ' ['FET-A-0', ro', I, ['FET-B-O', YOUT', r [' ('inductor-0', ''', 'OUT ]]
This minor correction preserves the original component arrangement and topology while resolving the node reference ambiguities, enabling successful simulation. This pattern indicates that the model effectively learns appropriate component placements and connectivity patterns but occasionally struggles with precise node labeling. These validity errors are therefore close valid designs and could be mitigated with improved node management during generation.
C.2. Efficiency Constraint Errors
In addition to validity, our generated circuits are evaluated against performance constraints such as minimum required efficiency, specified in the generation prompt. These constraints are verified post-hoe using SPICE simulation. While the majority of the model outputs achieve closely approximate the requested efficiency thresholds, few eircuits fall short. Consider the following example, where the prompt required an efficiency greater than 0.7:
[[' iniicctor-0', 'I'' '6'1, ['FE--B-O', rouT', ' I' I'cppccioo--1 0', '0', ), ['FET-A-O', YOUT', II
This topology, under generated duty cycle of 0.1, achieved a simulated efficiency of 0.625, missing the constraint by relatively small margin. The component usage and connectivity suggest that the model has captured many of the structural features that contribute to efficiency, although precise performance can depend on subtle circuit-level interactions. Such cases demonstrate that the model generates solutions near the constraint boundary and could be improved further with targeted refinement.
However, there also exist a few outlier cases where the efficiency gap more pronounced, such outputs scoring well below the target (for example, less than 0.3 when the required efficiency was greater than 0.7). These larger discrepancies typically associated with difficult trade-off's between duty cycle, output voltage, and interactions among other components. In these cases, the model may prioritize satisfying voltage or topological structure over efficiency, reflecting the inherent challenge of simultaneously satisfying multiple constraints. Nevertheless, the proximity of many failing cases to the desired efficiency threshold, along with the model's 's ability to produce performance-awar topologies, supports the conclusion that these errors arise from nuanced constraint balancing rather than fundamental model shortcomings.
D. Runtime Complexity Analysis
A key strength of AUTOCIRCUIT-RI. lies in its computational efficiency, particularly when compared with traditional search-based topology synthesis methods. These traditional methods, such as genetic algorithms and tree-based search (Fan et al., 2021; Zhao & Zhang, 2020), often rely on hundreds of SPICE simulations per design iteration. As a result, they typically require several minutes (hundreds seconds) to produce single valid circuit design, especially when adapting to new specifications or exploring diverse topologies.
14

---

AUTOCIRCUIT RL: Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation
In contrast, our method significantly reduces synthesis time by leveraging a two-phase RL approach. The LLM quickly generates initial candidate topologies, while the RL-based refinement performs iterative optimization using learned reward model that captures circuit validity, efficiency, and output voltage. On average, AUTOCIRCUIT-RL generates a complete and optimized circuit in approximately 2-3.5 seconds using two NVIDIA V100 GPUs, offering over 50x improvement in design time over traditional approaches.
To assess how the choice of language model affects runtime, we evaluated AUTOCIRCUI--LL with two LLMs: MPT-7B and LLaMA-3 8B. As expected, model size influences generation latency. The MPT-7B variant achieves faster runtimes (2.4-3.5 seconds for 4-10 component circuits), whereas the LLaMA-3 8B variant incurs slightly higher runtimes (2.8-5 seconds), reflecting the increased computational overhead of the larger model. Despite this, both configurations maintain runtimes well below traditional baselines, making them viable for real-time or interactive design applications.
Compleaity: 4Dum 3
using LlamA-3
Burnroe per Clreu (sacindd)
0.0
Number of Components
Figure Runtime Complexity of AUTOCIRCUITRRL Using MPT-7B vs. LLaMA-3 8B. The runtime per circuit increases with number of components but remains significantly faster than traditional scarch-based methods. MPT-7B offers lower latency due to its smaller model size, while LLaMA-3 8B provides marginally higher runtimes due to increased model capacity.
These results demonstrate that AUTOCIRCUIT-RE RL achieves an effective balance between computational cost and output quality, offering scalable and practical alternative to traditional methods in analog circuit synthesis.
15


Reviews:
[{"review_id": "3tyfl9MGUi", "reviewer": "Reviewer_ixgq", "summary": "The paper introduces AUTOCIRCUIT-RL, an RL-based LLM framework for automating analog circuit synthesis. The framework operates in two phases: instruction tuning and RL refinement. The authors claim that AUTOCIRCUIT-RL outperforms existing baselines, generating ~12% more valid circuits, improving efficiency by ~14%, and reducing duplicate generation rates by ~38%.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s constructive feedback and address each concern below.\n\n## Comparison with Auto-SPICE [1]:\n\nOur work fundamentally differs from Auto-SPICE, which focuses on generating SPICE netlists via domain-specific prompt engineering for dataset creation. It aims to generate large-scale SPICE netlist datasets from analog textbooks by annotating netlist schematics. In contrast, our approach—AutoCircuit-RL—goes beyond merely creating a dataset. It employs a two-phase process: first, an instruction-tuned LLM generates initial circuit topologies; then, a reinforcement learning (RL) refinement phase iteratively optimizes these topologies for multiple objectives (validity, efficiency, and expected output voltage). This reward-driven optimization enables our method to produce more robust designs and scale to more complex circuits.\n\nOur work differs from Auto-SPICE in several key aspects. Auto-SPICE focuses on automating the creation of large-scale SPICE netlist datasets (e.g., Masala-CHAI) by extracting netlists from schematic images and their captions, and then uses these datasets to fine-tune LLMs for SPICE netlist generation. In contrast, our approach—AutoCircuit-RL—is centered on the methodology of generating circuit topologies that adhere to specific design constraints using a synthetic dataset. We first leverage instruction tuning to generate initial topologies based on various user prompts, and then apply reinforcement learning to iteratively refine these designs with respect to objectives such as circuit validity, efficiency, and output voltage.\n\nWhile our methodology could potentially benefit from integrating datasets like those produced by Auto-SPICE, doing so addresses a different problem: dataset creation versus constrained topology synthesis. Our work tackles the challenge of generating circuit topologies under diverse, constraint-specific scenarios—a distinct and complementary problem to that addressed by Auto-SPICE.\n\n## Dataset Details:\nWe plan to release the code and dataset upon acceptance to support reproducibility. We have provided details of dataset collection in Section 2. Each data sample will include a circuit netlist representing the topology, along with its corresponding duty cycle, output voltage and efficiency obtained from the simulator.In the paper, “unique netlists” refer to differences in connectivity and node indexing, while “unique designs” abstract away superficial ordering variations. We will provide a representative sample subset for illustration.\n\n## Suggestions:\n\n### Qualitative Examples and Evaluation:\nThank you for the suggestion. We have included qualitative examples in the Appendix that showcase generated circuit topologies and highlight successful cases. These examples illustrate how our model adapts to design constraints and generates diverse solutions. We plan to expand these qualitative evaluations further in the final version.\n\n## Questions:\n\n### Computation Cost:\nOur method generates a circuit design in approximately 1.7 seconds using 2 NVIDIA V100 GPUs—substantially faster than traditional search-based methods, which often require hundreds of seconds due to extensive SPICE simulations. This efficiency is a direct result of our two-phase, reward-driven RL refinement process.\n\n### Comparison with Other Baselines:\nWe appreciate the suggestion to compare our approach with recent methods such as AnalogCoder and Artisan. While our primary comparisons have been with GraphVAE and other LLM-based methods, additional analyses show that our RL refinement significantly improves circuit validity and efficiency compared to these approaches. AnalogCoder relies on training-free code generation, which limits its ability to explore novel or complex topologies, whereas our method’s iterative refinement better addresses the multi-objective challenges of power converter design. AnalogCoder generates amplifier circuits while this work generates power converter topologies, it’s not a direct comparison.\n\n### Failure Cases and Detailed Prompts:\nWe recognize that including failure cases and error analysis will provide a clearer understanding of our approach and highlight potential areas for future improvement. While Table 3 currently presents examples of successful generation scenarios, we will expand it to include failure cases as well. Additionally, Table 3 in the Appendix provides detailed prompts for different constraint-related scenarios, offering insights into the model’s behavior under various conditions. These analyses help diagnose existing limitations and inform future refinements. We will incorporate further discussion on these aspects in the final manuscript upon acceptance.\n\n### Learning Curve of RL Tuning:  \nThe convergence curve of RL tuning will be added in the supplementary section upon acceptance.\n\n### Code Implementation:\nWe will opensource the code if this manuscript is accepted.", "claims_and_evidence": "The claims are supported by empirical evidence.", "methods_and_evaluation_criteria": "The evaluation is generally well-suited. However, the paper could benefit from more diverse benchmarks, or open-source benchmarks.", "theoretical_claims": "There is no theoretical claim.", "experimental_designs_or_analyses": "Experiments are generally sounds. \nThe learning curve of RL method is missing.", "relation_to_broader_scientific_literature": "This paper is about LLM-based circuit synthesis. Some relevant literature includes AnalogCoder, CircuitSynth, LaMAGIC.", "essential_references_not_discussed": "Can the author discuss the difference between the paper [1], which leverages llm for generating SPICE netlist?\n\n\nBhandari, Jitendra, et al. \"Auto-SPICE: Leveraging LLMs for Dataset Creation via Automated SPICE Netlist Extraction from Analog Circuit Diagrams.\" arXiv preprint arXiv:2411.14299 (2024).", "other_strengths_and_weaknesses": "Weakness:\n1. The dataset is not open-source, so it would be difficult to assess the performance fairly. \n2. More details regarding the dataset should be provided, for example, how does the dataset look like? What belongs to unique designs and unique netlists? What are some valid/invalid samples?", "other_comments_or_suggestions": "1. I suggest the author provide more qualitative examples and qualitative evaluation to demonstrate their methods and insights.", "questions_for_authors": "1. What is the computation cost of the proposed method?\n2. Can the authors provide a performance comparison against AnalogCoder?\n3. Can the authors provide some failure cases generated by the method?\n4. What are the detailed prompts used in the proposed method?\n5. Can the authors provide the learning curve of RL tuning?\n6. Can the authors provide their code implementation?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZWsP4uFlYW", "reviewer": "Reviewer_K9vp", "summary": "The authors proposed a novel framework, AUTOCIRCUIT-RL (AC-RL), for automating analog circuit topology synthesis using reinforcement learning (RL). The architecture operates in two main phases: instruction fine-tuning and RL refinement. Through extensive experimental validation, the authors demonstrated that the proposed framework outperforms previous research efforts, achieving significant improvements in (a) valid circuit generation, (b) efficiency in designing complex circuits, and (c) strong generalization capability with limited data.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. \n\n### Benchmarking Datasets:\nTo the best knowledge of the authors, currently in the domain of analog circuit topology generation, there is no standard benchmarking datasets. All the recent publications in this domain would spend considerable effort in curating initial dataset. Our focus is on power converter synthesis—a domain with unique design constraints such as efficiency, output voltage, and component pool. We have domain experts in power converter design who helped us examine the evaluation setup and validity of the dataset. Our dataset is specifically curated to represent these constraints and ensure that our evaluation reflects real-world challenges in analog circuit design. This specialized dataset is critical for assessing generative performance in our targeted application area.\n\n### Evaluation of Key Metrics:\nOur study comprehensively evaluates correctness, efficiency, and scalability by comparing multiple LLM-based approaches with a non-LLM baseline (GraphVAE) and by contrasting zero-shot, in-context learning, and fine-tuning methods. These layered evaluations clearly demonstrate that our framework not only generates more valid circuits but also achieves significant improvements in efficiency and scalability.\n\n### Runtime Complexity Analysis:\nOur method generates a new design in approximately 1.7 seconds using 2 NVIDIA V100 GPUs, which is a significant improvement over traditional search-based methods that typically require hundreds of seconds due to large number of simulation queries. This efficiency underscores the practical advantages of our RL-based refinement approach. We acknowledge that further runtime complexity analysis against standardized datasets could provide additional insights and plan to include such analyses in our final manuscript upon acceptance.", "claims_and_evidence": "Yes, the authors have made a significant effort to validate their claims through an in-depth experimental strategy, supported by satisfactory results and analysis demonstrated in this manuscript. They have also provided a thorough explanation for each metric and strategy adopted to substantiate the effectiveness of the proposed framework compared to previous state-of-the-art (SOTA) research works/models/architectures.", "methods_and_evaluation_criteria": "Yes, the proposed method, benchmark dataset, and evaluation criteria/metrics are explicitly defined and are satisfactory for demonstrating the application. The experimental design is well-structured and methodologically sound, covering a broad spectrum of LLM-based generative methods for circuit synthesis. The comparative analysis with GraphVAE strengthens its validity.", "theoretical_claims": "Yes, the theoretical claims regarding the framework's design, implementation, performance evaluation, and overall generalizability and scalability have been substantiated through experimental validation and proof of correctness.", "experimental_designs_or_analyses": "Yes....... soundness/validity of experimental designs has been validated against, (a) comparative benchmarking -> The study compares multiple LLM-based approaches against a non-LLM baseline (GraphVAE), ensuring a comprehensive evaluation of generative performance. (b) evaluation of Zero-Shot vs. Fine-Tuned Models --> The research systematically investigates how LLMs perform without fine-tuning (zero-shot & ICL) vs. with fine-tuning (prompt tuning & vanilla fine-tuning), providing a layered understanding of model capabilities, (c) use of soft-computing --> by integrating soft prompt tuning, the study tests an efficient fine-tuning strategy that preserves the base model while learning task-specific adaptations.\n\nThe primary concern to be highlighted is that if standard benchmarking datasets (e.g., modern IC netlists) are not used, the generalizability of the proposed approach may be limited. Further investigation is required to quantify key metrics such as correctness, efficiency, and scalability for complex circuit topology generation. Additionally, a runtime complexity analysis is necessary to evaluate trade-offs between the framework’s computational efficiency and the quality of the generated circuits and benchmark against previous research works.", "relation_to_broader_scientific_literature": "The key contributions of this research manuscript are potentially significant to the broader research community. The proposed framework not only serves the purpose of analog circuit design using LLM+RL but could also be pivotal in any domain involving constrained topology, such as agile software development, digital circuit synthesis, 3.5D/chiplet design, and more. A more in-depth analysis is presented in the \"Impact Statement,\" which has been validated to demonstrate its potential significance.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Mentioned and discussed in previous sections as \"Methods And Evaluation Criteria*\" and \"Experimental Designs Or Analyses*\".", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. The primary concern to be highlighted is that if standard benchmarking datasets (e.g., modern IC netlists) are not used, the generalizability of the proposed approach may be limited. Further investigation is required to quantify key metrics such as correctness, efficiency, and scalability for complex circuit topology generation. \n2. Additionally, a runtime complexity analysis is necessary to evaluate trade-offs between the framework’s computational efficiency and the quality of the generated circuits and benchmark against previous research works.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "no ethical review concerns flagged."}}, {"review_id": "IZ8yvoL61j", "reviewer": "Reviewer_PNqR", "summary": "This work proposes a framework for automating analog circuit topology synthesis using RL. It has two phases. In the instruction tuning phase, LLM learns to generate initial circuit topology, ensuring feasibility under basic constraints. In the RL phase, LLM iteratively optimizes topologies using reward models, enhancing validity, efficiency and output voltage to meeting the requirements.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review. \n\n## Device Sizing and Evaluation Metrics: \nOur approach primarily focuses on generating circuit topologies, which includes selecting the devices and their interconnections. The device parameters (e.g., capacitor, inductor, and MOSFET sizes) are set based on standard design guidelines commonly used in power electronics (e.g., 10μF for capacitors, 10μH for inductors, and fixed MOSFET parameters). This decision allows us to isolate and optimize the topology generation task while maintaining consistency in evaluations. We plan to focus on optimizing device parameters in the future. The metrics reported in Section 4.2—including circuit validity, efficiency, and output voltage—are computed using these standardized component values. While our method does not dynamically generate device sizes, the reported metrics remain valid and meaningful as they reflect the performance of different topologies under fixed design constraints.\n\n## Comparison with AnalogCoder and LaMAGIC:\n\nWhile both AnalogCoder and LaMAGIC use LLMs for circuit synthesis, our approach differs in its methodology and optimization strategy:\n\n### AutoCircuit-RL (Ours): \nWe employ a two-phase process: (i) an LLM first generates initial circuit topologies through instruction tuning, and (ii) a reinforcement learning (RL) refinement phase iteratively optimizes these topologies using reward models that account for circuit validity, efficiency, and output voltage. This allows us to explore a much wider design space and generate more robust and scalable topologies, even with limited training data. Our empirical results show a 12–14% increase in circuit validity and a 14–16% improvement in efficiency, demonstrating the effectiveness of our RL-based refinement.\n\n### AnalogCoder: \nThis approach uses domain-specific prompt engineering to generate PySpice code for standard analog circuits. However, it relies on a predefined synthesis library, which limits its ability to explore novel or complex topologies. Moreover, it does not cover power converters that require specialized design constraints, i.e., efficiency, and output voltage.\n\n### LaMAGIC: \nSimilar to other LLM-based methods, LaMAGIC generates circuits using supervised fine-tuning with pre-trained models but does not incorporate reinforcement learning. As a result, it lacks an iterative refinement process that can optimize designs beyond their initial generation. Our reward-driven optimization provides a key advantage in enhancing both circuit validity and efficiency.", "claims_and_evidence": "The experiments are solid and comprehensive, which support the claim made very well.", "methods_and_evaluation_criteria": "Only a synthesized dataset is considered for evaluation. While I acknowledge that it is difficult to collect a large-scale analogy topology dataset for training, it is feasible to evaluate the proposed method on a real analog topology dataset. With that, we can demonstrate that the data synthesis pipeline used is a reasonable one and the model trained on it generalizes well.", "theoretical_claims": "There is no theoretical claim in this paper.", "experimental_designs_or_analyses": "It seems that the proposed approach only generates the devices and the connection between them. How is the device size set? If the device sizes are not set, could we get the metrics mentioned in Section 4.2?", "relation_to_broader_scientific_literature": "The definition of reward function mentioned in this work may have broader impact.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "No additional comments.", "other_comments_or_suggestions": "No additional comments.", "questions_for_authors": "Is it possible to compare the proposed approach with AnalogCoder[1] and LaMAGIC[2]?\n[1] https://arxiv.org/abs/2405.14918 \n[2] https://arxiv.org/abs/2407.18269", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XeIXXYq7Ot", "reviewer": "Reviewer_FmMQ", "summary": "This paper proposes a reinforcement learning (RL)-based framework, AutoCircuit-RL, for automated analog circuit topology generation. It consists of two main stages: instruction fine-tuning and reinforcement learning optimization. During the instruction fine-tuning stage, supervised learning techniques are used to fine-tune a large language model (LLM). The RL optimization stage further refines the topology generation process using RL with AI feedback (RLAIF).", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s detailed feedback and address the concerns as follows:\n\n## Scalability Concerns:\nOur experiments primarily focus on 4–5 component circuits, with a few-shot evaluation for 6–10 component designs using only 1,000 training samples. Although success rates drop for larger circuits due to increased complexity, this decline is an inherent challenge of scaling circuit design—not a fundamental limitation of our method. Notably, even for larger circuits, our model achieves over 60% success in generating valid designs. Moreover, our experiments show that employing a multi-sample generation strategy (quantified by our SuccessRate@m metric) substantially improves performance. For instance, using m=3 or m=5 yields average generation times of 5.1 and 8.5 seconds, respectively, while markedly increasing the probability of finding a valid design. This approach effectively mitigates performance drops without necessitating massive additional training data. We are actively exploring enhanced RL sampling techniques and efficient data augmentation strategies to further improve scalability.\n\n## Comparison with Other Baselines:\nWe appreciate the reviewer’s suggestion to compare our approach with recent methods such as AnalogCoder and Artisan. While both methods provide valuable contributions to LLM-based circuit synthesis, our work differs fundamentally in methodology and scope:\n\nAnalogCoder relies on training-free prompt engineering to generate SPICE netlists, but it lacks an adaptive optimization mechanism, making it less suitable for synthesizing novel or complex topologies. Moreover, it does not cover power converters that require specialized design constraints, i.e., efficiency, and output voltage.\n\nArtisan is specialized for operational amplifier design using domain-specific LLMs, whereas our approach is focused on power convertors, though they could be more generalizable across different analog topologies.\n\nOur method (AutoCircuit-RL) introduces a two-phase process:\n\na. Instruction-tuned LLM generates diverse circuit topologies.\n\nb. Reinforcement learning (RL) refinement optimizes the topologies for multiple objectives (validity, efficiency, and output voltage) through reward-driven feedback.\n\nThis novel dual-phase strategy allows us to tackle multi-objective design challenges in power convertors more effectively. Additionally, we plan to add more quantitative comparisons with additional baselines, to demonstrate that our approach achieves superior design quality and efficiency. We will include these updated results in the final version if this is accepted.\n\n## Reference Clarifications:\nThank you for pointing this out. We acknowledge the issue of duplicate references and will correct them in the final manuscript. We will also include additional relevant references (e.g., Atelier, AnalogXpert) to better situate our work within the broader literature.", "claims_and_evidence": "1. The topology search is restricted to ~10 components, which is impractical for real-world designs.  \n2. Supported components lack diversity (only NMOS, PMOS, inductors, capacitors; missing resistors, diodes, NPN/PNP transistors).  \n3. Fixed device parameters oversimplify circuit generation.", "methods_and_evaluation_criteria": "1. Unclear necessity of reward models:  \n   - No comparison with Ngspice's runtime efficiency (fast for small circuits), raising doubts about the need for three separate reward models.  \n2. Handcrafted reward parameters:  \n   - Reward design relies on manual parameter tuning, limiting scalability.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "1. Scalability issues:  \n   - Trained on 100,000 samples for 4–5 component circuits, but success rates drop significantly for 6–10 component designs.  \n   - Massive training data likely required for larger-scale circuits. \n 2. Lack of modern baselines:  \n   - No comparison with recent methods, such as [1] and [2].\n \n References:\n[1] Lai, Y., Lee, S., Chen, G., Poddar, S., Hu, M., Pan, D. Z., & Luo, P. (2024). Analogcoder: Analog circuit design via training-free code generation. AAAI 2025.\n\n[2] Chen, Z., Huang, J., Liu, Y., Yang, F., Shang, L., Zhou, D., & Zeng, X. (2024, June). Artisan: Automated operational amplifier design via domain-specific large language model. In Proceedings of the 61st ACM/IEEE Design Automation Conference (pp. 1-6).\n\n[3] Zhang et al., “AnalogXpert: Automating Analog Topology Synthesis by Incorporating Circuit Design Expertise into Large Language Models,” arXiv, 2024.", "relation_to_broader_scientific_literature": "1. LLM-based analog topology generation: The framework generates analog circuit topologies containing up to 10 components and introduces a customized RL fine-tuning method to optimize circuit quality and performance.  \n2. Granular evaluation during RL feedback: The analog topology evaluation process is decomposed into detailed steps, enabling generated circuits to simultaneously satisfy design requirements and incorporate performance metrics.", "essential_references_not_discussed": "[1] Shen et al., “Atelier: An Automated Analog Circuit Design Framework via Multiple Large Language Model-based Agents,” TechRxiv, 2024. \n\n[2] Zhang et al., “AnalogXpert: Automating Analog Topology Synthesis by Incorporating Circuit Design Expertise into Large Language Models,” arXiv, 2024.\n\n[3] Chen, Z., Huang, J., Liu, Y., Yang, F., Shang, L., Zhou, D., & Zeng, X. (2024, June). Artisan: Automated operational amplifier design via domain-specific large language model. In Proceedings of the 61st ACM/IEEE Design Automation Conference (pp. 1-6).", "other_strengths_and_weaknesses": "There are two same items in the reference list:\n\nFan, S., Cao, N., Zhang, S., Li, J., Guo, X., and Zhang, X. From specification to topology: Automatic power converter design via reinforcement learning. In IEEE/ACM International Conference On Computer Aided Design (ICCAD), pp. 1–9, 2021a.\n\nFan, S., Cao, N., Zhang, S., Li, J., Guo, X., and Zhang, X. From specification to topology: Automatic power converter design via reinforcement learning. In IEEE/ACM International Conference On Computer Aided Design (ICCAD), pp. 1–9, 2021a.", "other_comments_or_suggestions": "It is better to show several generated circuits.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents AutoCircuit-RL, a reinforcement learning (RL)-based framework for automated analog circuit topology generation. 

Most reviewers agree that the problem is well-motivated, the proposed approach is novel, and the empirical evaluations are both solid and comprehensive. The authors make great efforts and address most of the concerns of the reviewers during the rebuttal phase.

Overall, the paper is valuable for this community. Therefore, I recommend accepting the paper.
Decision: Accept (poster)