CircuitSense: A Hierarchical MLLM Benchmark Bridging Visual Comprehension and Symbolic Reasoning in Engineering Design Process
Authors: Arman Akbari
Jian Gao
Yifei Zou
Mei Yang
Jinru Duan
Dmitrii Torbunov
Yanzhi Wang
Yihui Ren
Xuan Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Mathematical Reasoning Benchmark
Circuit Benchmark
Symbolic Reasoning
Equation Derivation
Primary area: datasets and benchmarks

Abstract: Engineering design operates through hierarchical abstraction from system specifications to component implementations, requiring visual understanding coupled with mathematical reasoning at each level. While Multi-modal Large Language Models (MLLMs) excel at natural image tasks, their ability to extract mathematical models from technical diagrams remains unexplored. We present \textbf{CircuitSense}, a comprehensive benchmark evaluating circuit understanding across this hierarchy through 8,006+ problems spanning component-level schematics to system-level block diagrams. Our benchmark uniquely examines the complete engineering workflow: Perception, Analysis, and Design, with a particular emphasis on the critical but underexplored capability of deriving symbolic equations from visual inputs. We introduce a hierarchical synthetic generation pipeline consisting of a grid-based schematic generator and a block diagram generator with auto-derived symbolic equation labels. Comprehensive evaluation of eight state-of-the-art MLLMs, including both closed-source and open-source models, reveals fundamental limitations in visual-to-mathematical reasoning. Closed-source models achieve over 85\% accuracy on perception tasks involving component recognition and topology identification, yet their performance on symbolic derivation and analytical reasoning falls below 19\%, exposing a critical gap between visual parsing and symbolic reasoning. Models with stronger symbolic reasoning capabilities consistently achieve higher design task accuracy, confirming the fundamental role of mathematical understanding in circuit synthesis and establishing symbolic reasoning as the key metric for engineering competence. Our synthetic pipeline code is available at \href{https://anonymous.4open.science/r/CircuitSense-8AC7/README.md}{URL}.

Published as a conference paper at ICLR 2026
A HIERARCHICAL MLLM BENCH- MARK BRIDGING VISUAL COMPREHENSION AND SYMBOLIC REASONING IN ENGINEERING DESIGN PROCESS
CIRCUITSENSE:
Arman Akbari 11, " Jian Gao', Yifei Zou', Mei Yang', Jinru Duan', Dmitrii Torbunov?,
Yanzhi Wang', Yihui Ren", Xuan Zhang'
Northeastern University, Brookhaven National Laboratory
ABSTRACT
Engineering design operates through hierarchical abstraction from system speci- fications to component implementations, requiring visual understanding coupled with mathematical reasoning at each level. While Multi-modal Large Language Models (MLLMs) excel at natural image tasks, their ability to extract mathe- matical models from technical diagrams remains unexplored. We present Cir- cuitSense, comprehensive benchmark evaluating circuit understanding across this hierarchy through 8,006+ problems spanning component-leve schematics system-level block diagrams. Our benchmark uniquely examines the complete engineering workflow: Perception, Analysis, and Design, with particular em- phasis the critical but underexplored capability of deriving symbolic equations from visual inputs. We introduce a hierarchical synthetic generation pipeline con- sisting of grid-based schematic generator and block diagram generator with auto-derived symbolic equation labels. Comprehensive evaluation of eight state- of-the-an MILMs, including both closed- source and open-source models, reveals fundamental limitations in tisatl-tt-mmattematiiall reasoning. Closed-source mod- els achieve over 85% accuracy on perception tasks involving component recogni- tion and topology identification, yet their performance on symbolic derivation and analytical reasoning falls below 19%, exposing a critical gap between visual pars- ing and symbolic reasoning. Models with stronger symbolic reasoning capabili- ties consistently achieve higher design task accuracy, confirming the fundamental role of mathematical understanding in circuit synthesis and establishing symbolic reasoning as the key metric for engineering competence.
Project page: /etps://cirruittee wwwpuutttteese---eennnnna......e wtteiiaaar..iiii........ tuub. io
INTRODUCTION
Mathematical modeling forms the foundation of all engincering disciplines. Mechanical engineers derive equations of motion to predict system dynamics (Shabana) 2005); optical engineers com- pute ray transfer matrices to design lens systems (Saleh electronic engineers to translate circuit schematics into symbolic transfer functions to analytically examine the different aspects the circuit performance (e.g., noise, stability, sensitivity, etc.). For example, phase-locked loop (PLL) with insufficient phase margin will oscillate, destroying func- tionality of the entire integrated electronic system, yet this catastrophic failure can only be predicted through mathematical analysis of the feedback network's 's poles and zeros (Hanumolu et al. 2004). Across these domains, the ability to translate visual representations such as circuit schematics, op- tical layouts, or system diagrams into precise mathematical formulations determines engineering success. This
2007). It is general practice for
reasoning represents a fundamental capability that no current AI system can replicate. However, unlike geometry or physics problems that operate in single
"Conespondence to akbari atheorhheeeer

---

Published as a conference paper at ICLR 2026
representational space, engineering uniquely requires this mathematical translation across multiple levels of hierarchy, from components to subsystems to complete architectures.
While Multi-modal Large Language Models (MLLMs) excel visual perception tasks, they exhibit a critical limitation: the inability to derive symbolic equation from visual representations (Liu et al.
2025
Lu et al. 2021 Pan al. 2025). This failure is not merely technical but fundamental: equation derivation distinguishes true engineering comprehension from pattern matching. Existing visual circuit benchmarks (Skelic al./ 2025 Shi et al. 2025 Li al. 2025 Zhao et al.) 2025), focus primarily on recognition- tasks like identifying component types, answering basic multiple- choice questions, or performing shallow numerical calculations. The core capability that defines circuit understanding remains untested: the ability to extract mathematical relationships from visual circuit topology that consistent across multiple system hierarchies.
We focus on analog circuits as a particularly rich domain for evaluating Souul-to-mathemen
capabilities. The analog design process progresses through multiple stages: topology creation (de- termining device types and interconnections) (Lai et al., 2025 |Chang et al. [2024] Dong et al.) 2023).
device sizing (optimizing physical dimensions for performance) (Wang et al. |020 Lyu et al. 2017
Cao et al. 2024), and layout design (Xu et al. 2019; Kunal et al. 2019 Crossley et al. 2013) (rep- resenting circuit as geometric shapes and physical layers for fabrication), with each stage building upon the previous. The design process in analog circuits suffers from long cycles where catastrophic failures like instability, oscillation, and excessive noise often remain hidden until final verification stages. The key accelerating analog design lies in early mathematical analysis.
This preventive approach relies on translating visual schematics into mathematical models. For ex- ample, for low frequency circuit design such as operational amplifier (Op-Amp), engineers derive equations to predict frequency response (Kamath et al., 1974), input and output referred noise (Hill- brand & Russer 2003), ensure stability margins, and optimize performance trade-off. For radio frequency circuits such as low noise amplifiers and power amplifiers, people derive the circuit's 's in- put and output impedance to ensure impedance matching and optimal power transfer (Wang et al.) 2010 Nguyen et al.) 2004). Yet no existing benchmark evaluates whether AI systems possess this circuit understanding and symbolic reasoning capability. Without examining the symbolic derivation process, we cannot assess whether models truly understand circuits or merely memorize visual pat- terns, and consequently, whether they can genuinely assist human designers in accelerating design eycles and catching critical failures before costly fabrication. This gap prevents from determin- ing if MLLMs are ready to serve as engineering tools or remain sophisticated but superficial pattern matchers.
Moreover, circuit design inherently operates across multiple levels of abstraction, requiring engineers seamlessly navi- gate between system architecture and component implementa- tion. Engineers typically begin with high-level block diagrams to architect complex systems such as analog- -oooogggtt-iiiall convert- ers (ADCs), phase-locked loops (PLLs), or multi-stage opera- tional amplifiers. They then systematically decompose these architectural blocks into component-level schematics, imple- menting each functional block using transistors, op-amps, and passive elements. Figure Iillustratss this hierarchical decom- position through a PLL example, showing how system-level blocks translate into transistor-level implementations. Ap- pendix A.3 provides detailed analysis questions for both this PLL and a two-stage op-amp, demonstrating the multi-level reasoning required for comprehensive circuit understanding. Despite the critical importance of this hierarchical reason- ing capability, no existing benchmark evaluates the ability to bridge between block diagrams and circuit schematics.
System Level
Kreo
N
Block Level AJ10
RLC
Am
Transistors
Figure 1: Multi-level hierarchy of Phase Lock Loop Design.
To fill this gap, we propose CireuitSense, the first benchmark that systematically evaluates circuit understanding through hierarchical mathematical reasoning. CireuitSense comprises 8,006 prob- lems organized across six hierarchy levels from resistor networks to system-level block diagrams with open-ended and multiple-choiee formats, testing three task categories that mirror the engi- neering workflow: Perception, Analysis, and Design. Our benchmark combines 2,986 carefully
2

---

Published as a conference paper at ICLR 2026
all Hierarchy Level
Level
Level
Level
Level
Resistor
RLC
Signal
Transistor
Block
System
Task Category
Perception
Analysis
Design
Noise
Hierarchical design
Vagmm WATRB: oz a
Schematic-lvvel design
design
System block isenttiaaiinn
Power Erergy
Risi
C16)
Pang Vene Lrma cos
Frequincy Rasponan
[[jي]] 11v11+ (w//we)))
Open end design
Design feetiback op
gains, unity goin (PMl6or) pf, moeting Ag110 68W15 slr (0.3-1 kHzz).
Transfer Function
Sources
Existing Online Resource
Hierorchical Synthetic Generation Pipelines
IEEE
arxiv iv
AM
Cisl
Google
Figure 2: Benchmark overview. CircuitSense evaluates circuit systems understanding across six hi- erarchy levels (resistor networks to system block diagrams), three task category (Perception, Analy- sis with equation derivation, and Design), using both curated problems and synthetically generated circuits systems with ground-truth symbolic equations.
curated problems from authoritative textbooks and documents with 5,020 synthetically generated circuits, uniquely emphasizing symbolic derivation. We introduce hierarchical synthetic gener- ation pipeline consisting of circuit schematic generator with guaranteed symbolic ground-truth equations, and a block diagram generator with symbolic transfer function ground-truth. This dual approach ensures both component-level depth and system-level breadth while preventing dataset contamination.
As illustrated in Figure 3, we evaluated CircuitSense over state-of-ther toffthe-ane art MLLMs and Gemini-2.5. Pro Google DeepMind 2025) showed the best performance among all tasks. Our main contribution and findings can be summarized as:
First Multi-Level Visual-to-Analltical Benchmark: We introduce the first benchmark that systematically evaluates understanding across engincering abstraction levels, from system-level block diagrams to component-level schematics, testing how models connect visual patterns different scales to their mathematical representations.
We developed two synthetic generation pipeline producing samples with guaranteed ground-truth equations: (i) component-level circuits with controlled complexity progression, and (ii) system-level block diagrams with hierarchical feedback structures, enabling isolated evaluation of visual comprehension and mathematical reasoning each abstraction level.
Hierarchical Synthetic Generation Pipeline:
Extensive Multi-Scale Performance Analysis: Through systematic evaluation of eight state-of-the-art MILMSs and detailed analysis of derivation attempts, we demonstrate that while closed-source models achieve over 85% accuracy on perception tasks, they fail catas- trophically at symbolic analysis (below 19% accuracy), with specific bottlenecks identi-
3

---

Published as a conference paper at ICLR 2026
fied including systematic output impedance misinterpretation and algebraic manipulation errors. Our experiments confirm that models with stronger equation derivation capabili- ties consistently achieve higher design task performance, establishing mathematical under- standing as prerequisite for Al-assisted circuit synthesis.
2 CIRCUITSENSE
In this section we introduce CircuitSense, a comprehensive visual benchmark consisting of 8,006+ problems for evaluating visual circuit understanding across different task categories and abstraction levels. CircuitSense evaluates visual circuit understanding through a hierarchical framework that mirrors the complete engineering design process, from high-level system architecture to detailed component implementation. As shown in Figure 2 the benchmark is organized along two primary axes: task categories and hierarchy levels. The dataset spans three task categories: Perception (890), Analysis (7043), and Design (157), with Analysis comprising the majority of problems as it directly tests the critical capability of extracting mathematical models from visual circuits. Problems are distributed across six hierarchical levels from basic resistor networks to system-level block diagrams, enabling fine-ge ane-grained assessment of where mothmmmmtiit translation fails.
2.1 DATA COLLECTION
We gather 2,986 curated problems for from authoritative sources ensure broad topical coverage across CMOS analog, RLC network analysis, and system-level circuit design. For Analysis task, our collection drew from two primary categories: (1) canonical textbooks widely adopted in undergrad-
uate and graduate curricula including Gray (2009); ; Razavi (2017): Allen & Holberg (012)) Bruun
Andebii-Andeebli 2022); Salam Rahman (2018); (2) university course repositories, including
University of Toronto ECE331 (Analog Electronics), Georgia Tech ECE6412 (Analog Integrated
Circuit Design), and Georgia Tech ECE3050 (Analog Electronics). For Perception questions we used a subset of circuit images from AnalogGenie (Gao et al. 2025 and our hierarchical synthetic generation pipeline. For Design task, we collected data from canonical analog circuit design text- books such as Gray (2009); Razavi 2017);[Aleer & Holberg (2012); Bruun along with representa- tive problem sets curated from university courses and design problems from ZeroSim (Yang et al. 2025). More detailes are provided in Appendix [A.4
However, curated problems suffer from potential dataset contamination and rarely test equation derivation systematically. To ensure unbiased evaluation, we developed hierarchical synthetic generation pipeline producing novel circuits with guaranteed ground-truth equations across different hierarchy levels, detailed in Section 2.2
2.2 HIERARCHICAL SYNTHETIC GENERATION PIPELINE
Circuit Schematic Generator Our schematic generator extends the MAPS framework (Zhu et al., 2025ab for Linear Pure Resistive Circuits (LPRC) to support the full spectrum of analog components. We construct circuit
X grid where dimensions Distribution to ensure topological diversity. Component selection follows a hicrarchical probability distribution that differs between inner and outer edges. We support 18 component types organized by complexity: passive elements (R, L. C), sources (voltage, current), controlled sources (VCVS, VCCS, CCVS, CCCS), and active devices (ideal op-amps). We treat the ideal op-amps as template subcircuits (input resistor, feedback network, and high-gain VCVS) and randomly place the whole template the grid.
sampled from Discrete Probability
The generator enforces electrical validity through multiple constraints: eliminating floating nodes, ensuring least d connectivity for all nodes, and maintaining exactly one voltage source per circuit to guarantee well-defined reference. The grid topology is translated into SPICE-compatile netlists through systematic node labeling and component enumeration. Circuit validation occurs at three levels: topological verification ensures no shorted components and proper control relation- ships, SPICE simulation confirms DC operating points and AC responses, and symbolic analysis through Leapy (Hayes) 2022) extracts ground-truth transfer functions (s)) (s)//V. and nodal equations via Modified Nodal Analysis. To manage computational complexity, we imple-
4

---

Published as a conference paper at ICLR 2026
Table I: Benchmark statisties.
Heracchical
Resporus
Task
Subcategory
Count
Design
Perception
806
Block-Lave
Transfer Function Analysia
Component Detection
200
Design
Connection Identification
200
Function classification
406
Schematic- Level Design
Small
Andlysis
Analysis
7043
Frequency Response
184
Transient Response
3811
Transfer Function Analysis
1736
Clossification
PBRR
Small Signal Analysis:
915
CMR & PSRR
54
Connection Mentification
Noise Jitter
Noise & Jitter Analysis
121
Analysis
Power Energy Analysis
222
Component Detection
Energy Analysis
Design
157
CPT-46
Owerd S-LL.
Sehematic-level
63
GL384EV
Block-level
56
GFT-do-eii
Gamnus
Hierarchical
38
Figure 3: Result of 8 representative MLLMs on Perception, Analysis, and Design tasks.
Total
8,006
ment adaptive timeouts based on circuit complexity scores, bypassing symbolic analysis for circuits exceeding practical computation limits.
Block Diagram Generator The block diagram pipeline constructs control systems through two- phase approach that ensures both structural validity and mathematical consistency. We begin by constructing main signal path consisting of E [To, Te] components, selected from library of standard transfer functions, placed sequentially along fixed horizontal axis. Components include transfer function blocks and summing junctions (with randomly assigned sign conventions for each input port), selected with probability [a:b], enabling both positive/negative feedback and feedfor- ward configurations. System complexity increases through systematic addiction of feedback and feedforward paths, with f2 [0, Ty.] feedback loops and nyf E [0, Tys) feedforward paths, with the algorithm preventing duplicate connections through set-based tracking. Each auxiliary path has probability Phlock 0.5 of containing an intermediate block, generating diverse architectures from simple unity feedback to complex multi-loop systems found in industrial applications such as ADCs and PLLs.
We compute the overall system transfer function using Mason's 's gain formula (Mason 1953). which systematically handles multiple feedback loops and forward paths. The algorithm identifies: all for- ward path P& from input to output, all loops L, in the system, and non-touching loop combinations. The system determinant A = 1 EL, + LL EL, Ly.. + ... computed symbolically, wehere the sum includes all combinations of non-touching loops. The overall transfer function be- comes (s)) = IPa.Ax, where A& is the determinant excluding loops that touch forward path Px. This approach correctly handles complex topologies inclusing nested loops.
This hierarchical approach to synthetic generation ensures comprehensive coverage from low-level component interactions to high-level system behavior, providing the multi-scale evaluation neces- sary for assessing true circuit understanding.
2.3 BENCHMARK STATISTICS
CircuitSense comprises 8,006 problems organized across three primary task categories that mirror the engineering workflow: Perception (890 problems), Analysis (7,043 problems), and Design (157 problems). As shown in Table [] the Analysis task dominates our benchmark, reflecting the cen- tral importance of mathematical reasoning in circuit understanding. Within Analysis, we balance 2,023 curated problems with 5,020 synthetically generated circuits to ensure both educational valid- ity and protection against dataset contamination. The distribution across subcategories reflects emphasis on equation derivation capabilities: Transient Response (3,811 problems) and Transfer Function Analysis (1,736 problems) are significantly larger categories because synthetic gen-
5

---

Published as a conference paper at ICLR 2026
eration pipeline primarily produces circuits for testing these fundamental mathematical skills. The remaining subcategories provide comprehensive coverage of circuit analysis: Small Signal Analysis (915) tests linearization and AC modeling, while Power & Energy Analysis (222), Frequeney Re- sponse (184), Noise & Jitter Analysis (121), and CMR & PSRR (54) evaluate specialized analytical skills.
Perception tasks comprise three subcategories: Component Detection (200 problems) tests whether models can accurately count and identify circuit elements, Connection Identification (200 prob- lems) evaluates netlist conversion capabilities to verify structural understanding of circuit topology, and Function Classification (406 problems) assesses whether models can infer circuit purpose from visual inspection alone. Together, these tasks establish whether models possess the visual compre- hension necessary for subsequent mathematical analysis. Design tasks progress through increasing levels of hierarchy: from schematic-level (63) and block-level design (56) to hierarchical design (38) that requires coordinating multiple abstraction levels.
Problems are organized across six complexity levels that capture the natural progression of cir- cuit design: Level 0 (Resistive Networks, 1.777 samples) for DC analysis, Level (RLC Circuits, 3,147 samples) for frequency-domain reasoning, Level 2 (Small Signal, 537 samples) with con- trolled sources, Level 3 (Transistor, 795 samples) for device-level analysis, Level 4 (Block, 559 samples) for operational amplifier abstraction, and Level 5 (System Diagrams, 228 samples) for system-level transfer functions. This hierarchical structure enables precise identification of where translation fails as complexity increases. Detailed level-specific statistics and problem distributions are provided in Appendix Table
3 EXPERIMENTS
In this section, we conduct comprehensive experiments to evaluate different closed-source and open- source MLLMs on CireuitSense across task categories and hierarchy levels. We test eight state-of- the-art models: Gemini-2.0-PH (Google DeepMind 2225)) Claude-4-Sonnet (Anthropic) 2025)
GPT-4o, GPT-4o-mini, Intern 7BB (Zhu al. (0225)) - 2025).
(Bai al.,
GLM-4.5V (Team et al. 2025b), and
(Team et al. 2025a).
3.1
EVALUATION FRAMEWORK
We employ two evaluation strategies depending on problem format. For multiple-choice questions, exact answer matching after standardized formatting. To enable multiple-choice evaluation on originally open-ended problems, we use Gemini-2.5-Flaas to generate three plausible distractor choices plus "None of the above" to avoid forcing random selection. For open-ended questions requiring numerical or short-form answers, we employ LLM-as-a-judge evaluation where Gemini- 2.5-Flash compares model responses against ground truth, accounting for equivalent representations and unit conversions, determining correctness based on mathematical equivalence rather than exact string matching. For design tasks that require simulations, we simulate them by Ngspice (ngspice Development Team 2024) with Skywater 130mm PDK (Sky Water Technology Foundry 2020).
Evaluating symbolic mathematical expressions presents unique challenges since a single equation
can be represented in numerous algebraically equivalent forms. For instance, Hs)) = 1/(RCs + +1) mathematically identical to H(s) = (1/RC)) RC))(ss ((+++(+1 1/RC)). To address this, we implement a rig- orous symbolic comparison pipeline using SymPy (Meurer et al., (017)) that performs: (1) parsing both predicted and ground-truth equations into symbolic expression trees, (2) algebraic simplifica- tion, (3) verification through symbolic subtraction, and (4) numerical validation by evaluating both expressions at 100 random complex frequency points when symbolic comparison is computation- ally intractable. This multi multi-pronged approach ensures robust evaluation even when models produce correct but differently formatted equations.
3.2
MAIN RESULTS
Perception Task We evaluated models on three perception subtasks: Component Detection, Con- nection Identification, and Function Classification. As Taalle][shows, closed-source models demon- strate strong visual understanding with over 86% accuracy, confirming that perception is not the

---

Published as conference paper at ICLR 2026
Table 2: Perception task results
Table 3: Design task results
Component Connection Function
Schematie- level(
Block- level %)
Hierarchical- Design( ())
Model
Model
Detec.(%)
Ident.
Class.(%)
GPT-4o
100
70
95
GPT-4o
10.52
36.36
18.92
GPT-4o-mini
90
24
GPT-4o-mini
30.64
36.36
32.43
Gemini-2.5-PP 2.r-Prr
100
100
95
Gemini-2.5 Pro
36.38
67.27
51.35
Claude-Sonnet-4
100
88
86
Claude-Sonnct
17.54
51.83
29.83
InternVL33-221
95
76
12
Intern VL3372B
7.01
52.73
29.73
Qwen2.5-VL
95
68
20
Qwen2.5-VL
8.76
30.91
18.92
GLM--45V
78
GLM-4.5V
50.91
32.35
Gemma-3-211 3-27B
90
24
68
Gemma-3 3--2BB
29.03
45,45
43.24
Table 4: Accuracies of different models on Analysis subcategories.
Transfer Function Analysis
Small Signal Analysis
Noise &
Power & Energy Analysis
Frequency Response
Transient Response
CMR PSRR
Model
Jitter Analysis
GPT-4o
52
16
43
37
50
42
GPT-4o-mini
41
9
36
16
47
Gemini-2.5-PP
83
13
38
74
77
90
87
Claude-Sonnet-4
64
23
66
64
73
67
Intern VL3-78L
15
8
18
12
17
20
Qwen2.5-V1--22B-nnss
40
14
31
18
37
38
GLM-4.5V
26
14
20
9
26
20
Gemma-3-227B
40
9
33
37
38
36
bottleneck for these systems. GPT-4o and Gemini-2.5-Pro achieve near-perfect performance (94- 100%), while Claude-Sonnet-4 maintains solid accuracy above 85%. In contrast, open-source mod- els struggle significantly with basic circuit structure recognition. For instance, GLM-4.5V achieves only 26% on Function Classification and 78% on Connection Identification, suggesting fundamental limitations in visual processing capabilities that precede any mathematical reasoning challenges.
Analysis Task We examined performance of models across Analysis subcategories. Table reveals that Gemini-22.5PPp dominates across all categories (13-90%), followed by GPT-4o and Claude- Sonnet-4 (6- (3%73%). while open-source models struggle significantly (below 40%). Furtheremore, models achieve higher accuracy traditionally complex tasks like Noise & Jitter Analysis (up to 90%) and Power & Energy Analysis (up to 87%) compared to fundamental tasks like Transient Response (3-13%) and Transfer Function Analysis (8-38%). This counterintuitive result occurs because our synthetic problems are concentrated these two fundamental subcategories, exposing the critical gap between memorized textbook solutions and genuine mathematical understanding. When models cannot rely on pattern matching from training data and must derive equations from novel circuits, their performance collapses dramatically. Section 4 separates synthetic and curated results to demonstrate how this weakness manifests when models cannot rely on memorized patterns.
Design Task Table reveals clear hierarchical pattern in design capabilities across all models. Models demonstrate significantly stronger performance at block-level design (30.91-67.27%) com- pared schematic-level design (7.01-36.38%). with hierarchical design falling between these ex- tremes. This pattern indicates that models can more readily manipulate abstract functional blocks than translate specifications into detailed component-level implementations. Notably, Gemini-2.5- Pro, which demonstrated superior symbolic equation derivation capabilities in the Analysis tasks, also dominates the Design tasks with 36.38% schematic-level, 67. 17.77% block-level, and 51.35% hi- erarchical design accuracy. This correlation between symbolic reasoning and design performance suggests that equation derivation capability serves as fundamental prerequisite circuit synthesis.

---

Published as a conference paper at ICLR 2026
Table Results of curated problems for Analysis task with multiple choice and open-ended format.
Model
Level 0 (Resistor)
Level (RLC)
Level 2 (Small Signal)
Level 3 (Transistor)
Level 4 (Block)
Overall
Accuracy
Multiple Choice Format (%)
GPT-4o
39.80
49.58
32.88
48.80
39.58
45.07
GPT-4o-mini
43.88
49.58
26.03
23.20
45.83
36.49
4
66.72
71.22
61.64
72.01
66.67
69.67
Gemini-2.5-Pro
74.04
87.39
78.08
81.72
89.58
80.71
Intern VL3-788
23.16
20.59
13.70
13.11
14.58
18.06
Qwen2.5-VLL-72b-ie..
29.53
41.60
30.14
35.94
29.17
34,90
GLM-4.5V
24.63
29.20
9.59
17.28
31.25
22.42
Comma--3-27
33.28
35.08
9.59
38.21
25.00
34.59
Open-ended format (%)
GPT-4o
29.59
29.83
19.18
13.96
17.81
22.84
GPT-4o-mini
44.44
38.24
32.88
18.62
31.25
32.09
Claude-Sonnet-4
35.56
50.21
12.33
27.04
33.33
34.76
Gemini-2.5-Pro
76.98
84.87
73.97
55.85
72.92
70.32
VL377nVL3-78B
20.79
19.54
6.85
14.47
10.42
17.26
Qwenn2.5-VI-72B-Insttcc
28.73
31.30
16.44
13.71
22.92
22.85
GLM-4.5V
34.44
39.71
13.70
19.50
25.00
28.83
Gemma-3-278
24.60
25.84
8.22
10.69
8.33
18.44
Table 6: Performance comparison on our hierarchical synthetic problems with symbolic equation
ground truth.
Model
Level (Resistor)
Level (RLC)
Level 2 (Small Signal)
Level 4 (Block)
Level 5 (System)
Overall
GPT-4o
1.50
3.33
5.80
7.33
9.65
4.98
GPT-4o-mini
3,20
3.80
7.54
3.40
0.88
3.76
Claude-Sonnet-4
2.83
5.16
5.80
11.64
7.89
6.29
Gemini-2.5-Pa
3.49
11.67
38.00
12.33
35.96
19.06
Intern VL3-78
1.50
3.67
6.68
3.72
0.44
3.50
Qwwen2.5-VIL772B-nnnnnn
0.83
4.17
6.03
6.64
10.09
4.96
GL.--455V
0.33
7.33
4.00
4.50
5.70
4.09
Gemma-3-27B
0.80
3.00
6.03
4.60
4.82
3.84
4 DISCUSSION
4.1 CURATED VS. SYNTIETTIC PERFORMANCE
While overall Analysis task performance provided initial insights, the aggregate 7,043-problem eval- uation masks critical patterns that emerge only through systematic decomposition. We conducted three complementary evaluations six-level hierarchy. First, we tested 2,023 curated textbook prob- lems in multiple-choice format, where models could leverage answer elimination strategies. Second, we evaluated same problems open-ended format, removing scaffolding of provided op- tions. Third, we assessed 5,020 synthetic circuits requiring direct equation derivation without any answer choices.
The three evaluation formats reveal systematic degradation in mathematical reasoning capability. As shown in Table in multiple-choice format, Gemini-2.5-Prr achieves 80.71% on curated problems and maintains 70.32% in open-ended evaluation. This 10- 10-point drop suggests some analytical abil- ity beyond elimination. However, other models collapse catastrophically without answer options, falling below 35% and exposing heavy reliance pattern matching. Table that synthetic circuits requiring equation derivation, it catastrophically fails at 19.06%, 61 wewpeeettee-poooi drop from multiple-choiee performance. Other models show even steeper degradation: Claude-Somne Sonnet-4 falls from 69.67% (multiple-choice) to 34.76% (open-ended) to just 6.29% (synthetic), while open-
8

---

Published as a conference paper at ICLR 2026
Table Performance Analysis on Gemini-2.5-Pro across transfer function derivation subtasks.
Subtask
Description
Acc. (%)
Component Identification
Identify all components
97
Impedance Assignment
Convert components s-domain impedance
95
Total Impedance Calculation
Compute equivalent input impedance
81
Output Impedance Derivation
Calculate the impedance at the output node
8
Impedance Ratio Formation
Apply voltage divider principle correctly Simplify to canonical form
39
Transfer Function Simplification
55
source models barely exceed 4% on synthetic problems. This systematic collapse confirms that models rely on answer elimination and pattern matching rather than mathematical reasoning.
Analysis across hierarchy levels reveals that models develop specialized capabilities rather than uni- form understanding. On curated problems (Table 5), different models excel at different abstraction levels. Gemini-2.5-Pro peaks at Level 4 (Block-level with ideal op-amps, 89. 89.58%), while Claude- Sonnet-4 achieves highest performance at Level 3 (Transistor circuits, 72.01 %). This specializa- tion pattern persists in synthetic evaluation (Table [ but with revealing differences: Gemini-2.5-PP achieves its best synthetic performance at Level 2 (Small Signal, 38.00%) and Level 5 (System-level block diagrams, 35.96%), while Claude-Sonnete peaks at Level 4 (Block-level, 1 11.64%).
4.2 FAILURE POINT ANALYSIS
To understand why models fail at equation derivation despite recognizing circuit components, we analyzed 100 transfer function derivation attempts by Gemini-2.5 decomposing the process into six sequential subtasks. As shown in Table 7 while 55% of attempts correctly computed total impedance, only 8% succeeded at output impedance derivation which is seemingly simpler task. This 47% drop represents the primary bottleneck the entire pipeline. The subsequent partial recovery to 39% in impedance ratio formation and 55% in final transfer function suggests the model sometimes reaches correct answers through different reasoning paths compare to human. The higher final accuracy compared to Table 4 reflects our selection of the first 100 questions, which proved easier than the full transfer function analysis subset. Additional system-level failure analysis is provided in Appendix/A.1 A.1
RELATED WORKS
Visual Reasoning in Multi-modal Language Models Recent advances in Multi-modal Large Language Models (MLLMs) (Bai et al. 2025 Zhu et al., 2025b) Google DeepMind] (025) have demonstrated remarkable progress in integrating visual and linguistic information, achieving strong performance on tasks like visual question answering. To evaluate these capabilities, several bench- marks have emerged focusing on visual mathematical reasoning. Most visual math benchmarks (Lu et al. 2024, Wang et al.) (025) evaluates mathematical reasoning in visual contexts but primarily tests knowledge-centric problems that can often be solved through pattern recognition rather than true mathematical understanding. Scientific diagram benchmarks including ScienceQA (Lu et al.) 2022), and SeePhy (Xiang et al.) (225)) extend evaluation to -opain-specific content, testing under- standing of physics phenomena. However, these benchmarks evaluate whether models can select correct answers or perform numerical calculations, but do not assess the fundamental capability of translating visual representations into formal symbolic mathematical expressions.
Visual Circuit Understanding Benchmarks Existing circuit-focused benchmarks severely un- derestimate the complexity of circuit analysis by focusing on shallow tasks within single abstraction levels. CIRCUIT (Skelic et al. 2025) present 510 RLC-only circuit questions. EEE-Bench Li et al. (2025) propose 2860 hand- picked visual questions across electrical and electronics engineering. MMCireuitEval Zhao et al. (2025) also curated 3614 question-ansee answer pairs for digital and analog circuits. Furthermore, AMSbench (Shi et al. 2025)) provides analog and mixed-signal circuit prob- lems but focuses on multiple-choice questions testing conceptual understanding. Critically, none of these benchmarks evaluate the fundamental capability of translating visual circuit representations

---

Published as a conference paper at ICLR 2026
into symbolie mathematical equations and also lack hierarchical organization that mirrors real en- gineering abstraction levels. CircuitSense addresses this gap by systematically evaluating visual comprchension and mathematical reasoning across the complete hierarchy.
CONCLUSION
We introduce CircuitSense, a comprchensive benchmark of 8,006 problems for evaluating visual-to- mathematical reasoning circuit understanding which combines curated questions with synthetic problems focused on symbolic equation derivation. Our hierarchical synthetic generation pipeline produces novel circuits across six levels with guaranteed ground-truth symbolic equations, enabling rigorous evaluation. Our extensive evaluation on perception, analysis, and design tasks shows that models demonstrate adequate perception (85%+ for closed-soures but fail catastrophically at math- ematical symbolic modeling (below 19%). This mathematical weakness directly undermines their design capabilities.
7 LIMITATION AND FUTURE WORKS
While CircuitSense advances circuit understanding evaluation, several limitations present opportu- nities for expansion. Our synthetic pipeline currently focuses on transfer function derivation and nodal analysis, missing other analysis types like noise analysis or frequency response. We plan to extend our symbolic generation pipeline to all subcategories. Computational constraints limit synthetic circuits to 12-15 components since symbolic equation derivation becomes prohibitively expensive beyond this scale, restricting our ability to test understanding of larger circuits.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our results, we provide comprehensive resources and documenta- tion. The complete CircuitSense dataset is included in the supplementary materials. Our hierarchi- cal synthetic generation pipeline code for creating additional circuits with guaranteed ground-truth equations is available at GitHub The exact model versions and inference parameters for all six evaluated MILMs are specified in Appendix[A.3 Furthermore, the llm-as-a-judge prompt template also provided in Appendix A.8 fro evaluation process.
ACKNOWLEDGMENTS
We thank anonymous reviewers for their valuable feedback. This project is partially supported by the National Science Foundation under awards CCF-2526432 and CCF-2416375, Yihui Ren and Dmitrii Torbunov were supported by the Laboratory Directed Research and Development Program of Brookhaven National Laboratory, which is operated and managed for the U.S. Department of Energy Office of Science by Brookhaven Science Associates under contract No. DF-SC0012704.
REFERENCES
AIC Design. Academic courses. httppp///a.ddeeii...rr////edeoep cessed: 2025-09--8.
Ac-
All About Circuits. All about cireuits. http::/pppwpwwwllooao.o......... Accessed: 2025-09-18.
P. E. Allen and Douglas R. Holberg. CMOS Analog Circuit Design. The Oxford Series in Electrical and Computer Engineering. Oxford University Press, USA, 3rd ed edition, 2012. ISBN 978-0- 19.976507-2.
Anthropic. Introducing claude httppp//www..ttooool..coo//////p 4 2025. Large language model.
10

---

Published as a conference paper at ICLR 2026
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu. Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang. Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. URL httptttsss//aii...tt///aptt
www.////..... 13923
Erik Bruum. CMOS Analog IC Design: Problems and Solutions.
Weidong Cao, Jian Gao, Tianrui Ma, Rui Ma, Mouhacine Benosman, and Xuan Zhang. Rose-opt: Robust and efficient analog circuit parameter optimization with knowledge-infused reinforcement learning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2024.
Chen-Chia Chang, Yikang Shen, Shaoze Fan, Jing Li, Shun Zhang, Ningyuan Cao, Yiran Chen, and Xin Zhang. Lamagic: Language-model- model-based topology generation for analog integrated circuits. arXiv preprini urXiv:2407.18266 2024.
Chegg Inc. Chegg. https://wwat chegg. com/
Accessed: 2025-09-18.
John Crossley, Alberto Puggelli, H-P Le, B Yang. R Nancollas, Kwangmo Jung, Lingkai Kong, Nathan Narevsky, Yue Lu, Nicholas Sutardja, et al. Bag: designer-oriented integrated frame- work for the development of ams circuit generators. In 2013 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 74-81. IEEE, 2013.
Zehao Dong, Weidong Cao, Muhan Zhang, Dacheng Tao, Yixin Chen, and Xuan Zhang. Cktgnn: Cireuit graph neural network for electronic design automation. arXiv preprint arXiv:2308. 16406, 2023.
Embedded Wala. Embedded wala.
Accessed: 2025-09-18.
Jian Gao, Weidong Cao, Junyi Yang, and Xuan Zhang. automatic discovery of analog circuit topologies, 2025. URL
Analoggenie: A generative engine for
25C3.00205
Google DeepMind. Gemini 2.5:
Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. URL
2507.06265
Paul Gray (ed.). Analysis and Design of Analog Integrated Circuits. Wiley, 5th ed edition, 2009. ISBN
Pavan Kumar Hanumolu, Merrick Brownlee, Kartikeya Mayaram, and Un-Ku Moon. Analysis of charge- pammcepmppppp phase-locked loops. IEEE Transactions on Circuits and Systems I: Regular Papers, 5165):665-1664 2004.
Michael Hayes. Leapy: symbolic linear circuit analysis with Python. PeerJ Computer Science, pp- c875, February 2022. ISSN 2376-5992, doi: 10.7717/peerj-cs.88
htLlattss///oo.....
10. 70.7711/pa ht111//peer-ccee cs. 875
Herbert Hillbrand and Peter Russer. An efficient method for computer aided noise analysis of linear amplifier networks. IEEE transactions on Circuits and Systems, (3(4):335-238, 2003.
BYT Kamath, Robert G Meyer, and Paul R Gray. Relationship between frequency response and settling time of operational amplifiers. IEEE Journal of Polid-State Circuits, (6))34773322 1974.
Kishor Kunal, Meghna Madhusudan, Arvind K Sharma, Wenbin Xu, Steven M Burns, Ramesh Harjani, Jiang Hu, Desmond A Kirkpatrick, and Sachin S Sapatnekar. Align: Open-source analog layout automation from the ground up. In Proceedings of the 56th Annual Design Automation Conference 2019, pp. 1-4, 2019.
Yao Lai, Sungyoung Lee, Guojin Chen, Souradip Poddar, Mengkang Hu, David Z Pan, and Ping Luo. Analogcoder: Analog circuit design via training-free free code generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 379-387, 2025.
11

---

Published as a conference paper at ICLR 2026
Learn Electronics India. Learn electronics httphhtttttee//.....aaaaaaeleeea com/ Accessed: 2025-09-18.
Ming Li, Jike Zhong, Tianle Chen, Yuxiang Lai, and Konstantinos Psounis. Eee-bench: A com-
prehensive multimodal electrical and electronics engineering benchmark, 2025. URL https:
//arxiv.org///// abs / /411 01492
Jiaqi Liu, Songning Lai, Pengze Li, Di Yu, Wenjie Zhou, Yiyang Zhou, Peng Xia, Zijun Wang, Xi Chen, Shixiang Tang, Lei Bai, Wanli Ouyang. Mingyu Ding, Huaxiu Yao, and Aoran Wang- Mimicking the physicist's eye:a vim-centric approach for physics formula discovery, 2025. URL
https::/arxiv.co wwtp://w.rrwoo/oo/oooooooooos///
17380
Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning, 2021.
wttp://////../.....1..01.. 04165
Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In The 36th Conference on Neural Information Processing Systems (NeurIPS), 2022.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannanch Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In The Twelfth International Conference on Learning Representations, 2024. URL httpp:://ppereeeettte
net http:///ppppprrooe.eeeee./eeeeea
Wenlong Lyu, Pan Xue, Fan Yang, Changhao Yan, Zhiliang Hong, Xuan Zeng, and Dian Zhou. An efficient bayesian optimization approach for automated optimization of analog circuits. IEEE Transactions on Circuits and Systems I: Regular Papers, 65(6))1954- 1967, 2017.
Samuel J. Mason. Feedback theory-some properties of signal flow graphs Proceedings of the Crw..o/psIoroooCC
IRE, 41:1144-1156, 1953. URL http::::/app.semattiiesseeeeeeee 17565263
Aaron Meurer, Christopher P. Smith, Mateusz Paprocki, Ondřej Čertík, Sergey B. Kirpichev, Matthew Rocklin, Amit Kumar, Sergiu Ivanov, Jason K. Moore, Sartaj Singh, Thilina Rath- nayake, Sean Vig, Brian E. Granger, Richard P. Muller, Francesco Bonazzi, Harsh Gupta, Shivam Vats, Fredrik Johansson, Fabian Pedregosa, Matthew J. Curry, Andy R. Terrel, Štěpán Roučka, Ashutosh Saboo, Isuru Fernando, Sumith Kulal, Robert Cimrman, and Anthony Scopatz. Sympy: symbolic computing in python. PeerJ Computer Science. 3:e003, Jamuary 2017. ISSN 2376-5992. doi:
cs. 103. [tttttss::///o http://......0101777//epeee--33-
ngspice Development Team. ngspice circuit simulator. net 2024. Version 43.
Trung- Kien Nguyen, Chung- -wan Kim, Gook-Ju Ihm, Moon-Su Yang, and Sang- Lee. Cmos low-noise amplifier design optimization techniques. IEEE Transactions on microwave theory and techniques, 52(5): 1eee:1433-444 1442, 2004.
Yicheng Pan, Zhenrong Zhang, Pengfei Hu, Jiefeng Ma, Jun Du, Jianshu Zhang, Quan Liu, Jianqing Gao, and Feng Ma. Enhancing the geometric cooblem-solving ability of multimodal 1lms via symbolic- neurall integration, 2025. URL
Mehdi Rahmani-Andebili. Advanced Electrical Circuit Analysis: Practice Problems, Methods, and Solutions. Springer International Publishing, 2022. ISBN 9777
6.
Behzad Razavi. Design of Analog CMOS Integrated Circuits. McGraw-Hill Education, second edition edition, 2017. ISBN 978-0-07--22293-2
Md. Abdus Salam and Quazi Mehbubar Rahman. Fundamentals of Electrical Circuit Analysis. Springer Singapore, 2018 ISBN
978-981-10-8624-2
12

---

Published as a conference paper at ICLR 2026
Bahaa Saleh and Malvin Teich. Fundamentals of Photonics, 2nd Edition. Wiley, 06 2007. ISBN 9780471358329.
Ahmed Shabana. Dynamics of Multibody Systems. Cambridge University Press, 01 2005.
Yichen Shi, Ze Zhang, Hongyang Wang, Zhuofu Tao, Zhongyi Li, Bingyu Chen, Yaxin Wang, Zhen huang, Xuhua Liu, Quan Chen, Zhiping Yu, Ting-Jung Lin, and Lei He. Amshench: A comprehensive benchmark for evaluating mllm capabilities in ams circuits, 2025. htthttps:
24138
Lejla Skelic, Yan Xu. Matthew Cox, Wenjie Lu, Tao Yu, and Ruonan Han. Circuit: A benchmark for circuit interpretation and reasoning capabilities of Ilms, 2025. URL https: org/ fbs/2502.07983
SkyWater Technology Foundry. SkyWater SKY130 Open Source Process Design Kit. https:
//github. com/ ctm googl e/ wwwps//wtttteradaeo... 2020. 130nm CMOS Technology.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Cas- bon, Etienne Pot. Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer. Xi- aohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Cole- man, Yi Gao, Basil Mustafa, Iain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan-Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju. Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky. Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng. Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang. Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlie Chen, Charline Le Lan, Christopher A. Choquette-Choo, CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng. Dimitris Paparas, Divyashree Shivaku- mar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eu- gene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Hlimezak-llucnnkke Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wi- eting. Jonathan Lai, Jordi Orbay. Joseph Fernandez, Josh Newlan, Ju yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Ni- lay Chauhan, Noveen Sachdeva, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Ruben- stein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanezyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq lqbal, Shashir Reddy, Shruti Sheth, Siim Pöder, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger. Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evci, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow. Yuvein Zhu, Zichuan Wei, Zoltan Fgyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins. Joelle Bar- ral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer. Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Jean- Baaanstt Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma technical report, 2025a. URL
aaaaaaiivvv... http:///wwwxw..gaa/////// /203 19786
Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng. Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang. Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang. Ziyang Pan, Aohan Zeng, Baoxu Wang, Bin Chen, Boyan Shi, Changyu Pang, Chenhui Zhang, Da Yin. Fan Yang, Guoqing Chen, Jiazheng Xu, Jiale Zhu, Jiali Chen. Jing Chen, Jinhao Chen, Jinghao Lin, Jinjiang Wang. Junjie Chen, Leqi Lei, Letian Gong, Leyi Pan, Mingdao Liu, Mingde Xu, Mingzhi Zhang, Qinkai Zheng, Sheng Yang, Shi Zhong,
13

---

Published as a conference paper at ICLR 2026
Shiyu Huang, Shuyuan Zhao, Siyan Xue, Shanggin Tu, Shengbiao Meng. Tianshu Zhang, Tianwei Luo, Tianxiang Hao, Tianyu Tong, Wenkai Li, Wei Jia, Xiao Liu, Xiaohan Zhang, Xin Lyu, Xinyue Fan, Xuancheng Huang, Yanling Wang. Yadong Xue, Yanfeng Wang. Yanzi Wang, Yifan An, Yifan Du, Yiming Shi, Yiheng Huang, Yilin Niu, Yuan Wang, Yuanchang Yue, Yuchen Li, Yutao Zhang. Yuting Wang. Yu Wang, Yuxuan Zhang, Zhao Xue, Zhenyu Hou, Zhengxiao Du, Zihan Wang, Peng Zhang, Debing Liu, Bin Xu, Juanzi Li, Minlie Huang, Yuxiao Dong, and Jie Tang. Glm-4.5v and 1.m-4.1vtthinking Towards versatile multimodal reasoning with scalable reinforcement learning, 2025b. URL https: / arxiv. org/ abs/ 2507 01006
Hanrui Wang, Kuan Wang. Jiacheng Yang, Linxiao Shen, Nan Sun, Hae-Seung Lee, and Song Han. Gen-rl circuit designer: Transferable transistor sizing with graph neural networks and reinforce- ment learning. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pp. 1-6. IEEE, 2020.
Hua Wang, Constantine Sideris, and Ali Hajimiri. A cmos broadband power amplifier with a transeormer-based high-order output matching network. IEEE journal of solid-state circuits, 45 (12):2709-2722,, 2010.
Ke Wang. Junting Pan, Linda Wei, Aojun Zhou, Weikang Shi, Zimu Lu, Han Xiao, Yunqiao Yang, Houxing Ren, Mingjie Zhan, and Hongsheng Li. Mathcoder- VL. Bridging vision and code for enhanced multimodal mathematical reasoning. In The 63rd Annual Meeting of the Associ- ation Computational Linguistics, 2025. URL nuvt X1 imAb
net ftffmormmmo ?id-
Kun Xiang, Heng Li, Terry Jingchen Zhang, Yinya Huang, Zirong Liu, Peixin Qu, Jixi He, Yu- Jie Yuan Jiaqi Chen, Jianhua Han, Hang Xu, Hanhui Li, Mrinmaya Sachan, and Xiaodan Liang. Seephys: Does seeing help Sanking2-benchmarkin vision-based physics reasoning. arXiv preprint arXiv:2505.1900 2025.
Biying Xu, Keren Zhu, Mingjie Liu, Yibo Lin, Shaolan Li, Xiyuan Tang, Nan Sun, and David Z Pan. Magical: Toward fully automated analog ic layout leveraging human and machine intelligence. In 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 1-8. IEEE, 2019.
Xiaomeng Yang, Jian Gao, Yanzhi Wang, and Xuan Zhang. Zerosim: Zero-shot analog circuit eval- uation with unified transformer embeddings. In Proceedings of the 44th IEEE/ACM International Conference on Computer-Aided Design, pp. 1-9, 2025.
Chenchen Zhao, Zhengyuan Shi, Xiangyu Wen, Chengjie Liu, Liu, Yunhao Zhou, Yuxiang Zhao, Hefei Feng. Yinan Zhu, Gwok Wwa Wan, Xin Cheng, Weiyu Chen, Yongqi Fu, Chujie Chen, Chenhao Xue, Guangyu Sun, Ying Wang, Yibo Lin, Jun Yang, Ning Xu, Xi Wang, and Qiang Xu. Mmcircuiteval: A comprehensive multimodal circuit-focused benchmark for evaluating llms, 2025. URL /Rt.tttss///ax....oo/////// www.aawwwww.rgaa/////a/aa 19525
Erle Zhu, Yadi Liu, Zhe Zhang, Xujun Li, JinZhou, Xinjie Yu, Minlie Huang, and Hongning Wang- MAPS: Advancing multi- modai-modal reasoning in expert-level physical science. In The Thirteenth International Conference Learning Representations, 2025a. URL URL /oppnnevvee.....
net net/fooumm? wtt.rrummmd-GoOOyF333333 Ipd
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang. Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li, Yinan He, Tan Jiang, Jiapeng Luo, Yi Wang, Conghui He, Botian Shi, Xingcheng Zhang, Wenqi Shao, Junjun He, Yingtong Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Han Lv, Lijun Wu. Kaipeng Zhang, Huipeng Deng, Jiaye Ge, Kai Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. In- ternv13: Exploring advanced training and test-time recipes for open-source multimodal models, 2025b. URL httpt::::///......... pttttp:://xxiie v. org. abs/ 2504. 0479
14

---

Published as a conference paper at ICLR 2026
Table 8: System level analysis
Models
Exact
High-level
GPT-4o
9.65
1.75
GPT-4o-mini
0.88
10.09
Claude-Sonnet-4
7.89
28.51
Gemini-2.5
35.96
39.04
Intern (itemnL3-77881
0.44
1.75
Dwen2.5-VIL-721
10.09
9.65
GLM--.5V
5.70
17.98
Gemma-3-27B
4.82
12.72
Table Analysis: Detailed Statistics by Level
Abstraction Levels
Curated Data
Synthetic Data
Level (Resistor)
631
1,146
Level (RLC)
476
2,671
Level (Small Signal)
73
464
Level (Transistor)
795
Level 4 (Block)
48
511
Level (System)
228
Total
2,023
5,020
A APPENDIX
A.1 BLOCK DIAGRAM TRANSFER FUNCTION ANALYSIS
Symbolic vs. Abstract Representations To understand how models handle system-level abstrac- tion, we analyzed their performance on block diagram transfer function derivation under three condi- tions: high-level representations using simplified block labels ("G1" " or "H,"", "), exact representations with complete transfer functions "10/(s + 5)". This comparison reveals whether models struggle with topological understanding of feedback systems with the algebraic manipulation required for symbolic computation.
Table demonstrates that algebraic complexity, topological reasoning, fundamentally limits model performance. All models show significant accuracy degradation when moving from high- level exact representations: Gemini-2.0-PP drops from 38%18% to 35.96%, while Claude-Sonnet- 3.5 exhibits a more dramatic decline from 28.51% to 7.89%. This consistent pattern reveals that models can successfully apply Mason's gain formula to abstract symbols but fail when manipulating complex rational functions with multiple terms. The performance gap indicates that current MLLMS possess adequate understanding of feedback topology and control theory principles but lack the symbolic mathematics capabilities essential for engineering analysis, suggesting that improvements should focus on enhancing algebraic reasoning rather than visual comprehension.
A.2 HIERARCHY STATISTICS
We classified each question according its component complexity, from Level 0 (resistor-only networks) to Level 5 (system-level block diagrams). Table(9presents the distribution across abstrac- tion levels for both curated and synthetic datasets within the Analysis task category. The curated data concentrates at the extremes-Level (631) and Level (795)-reflectin textbook emphasis on foundational concepts and transistor-level design. Notably, we have no curated Level prob- lems and no synthetic Level 3 problems, as system-leve textbook problems rarely require equation derivation while transistor circuits resist symbolic generation due their nonlinear device models.
A.3 DETAILED HIERARCHICAL EXAMPLES
We present two concrete examples of design questions that require answers derived from multiple levels of analysis. These examples show the application symbolic equation derivation across different hierarchy in engineering design process which motivated us to collect CireuitSense.
15

---

Published as a conference paper at ICLR 2026
Level
Level
Level
Figure 4: Step-by-step analysis of Two-stage Op-Amp
A.3.1 TWO-STAGE OP-AMP
Figure 4 illustrates how Levels through 2 can be applied to analyze two-stage Op-Amp's Gain transfer function, with each level yielding concrete analytical outputs At the block level (Level 4), the amplifier is identified as two cascaded gain stages with compensation capacitor, leading to high-level transfer expression shown below:
Van(
H(s) H()) - -Nc(ss) (s)
(I)
Where (s)1,,2 1,2 is the transfer function of corresponding Stage.Nce(s)) is the Zero factor introduced by feedback capacitor. Next, at the transistor level (Level 3), the actual circuit topology is consid- ered: the differential input pair with current-mirror load feeding into common-source second stage. At this level, At this level, key device parameters of stage and stage 2, such as the transconductance (gm) and output resistance (r.), can be derived:
0lp
Gm1,2 @VGs
(2)
To1,2 R
(3)
AID
Where, In is the drain current; VGs is the gate-to-source voltage; À is the channel-length modulation parameter. Once the key parameters and transistor structure are clear, it can be abstracted into small-signal equivalent (Level 2), each MOSFET is replaced by its hybrid-n model, reducing the circuit dependent sources, ro, and parasitic capacitance. After involving the pole and zero effects, the final transfer function can then be written explicitly as:
(s) (8)
9m1Fol 9m2Noo w,
(1
H(s) H(8) Noe(s) As
(4)
(1
wp1
This flow connects qualitative structure identification quantitative expressions, ensuring that the final analysis yields concrete transfer function.
A.3.2 PHASE-LOCKED LoOP
Another example of hierarchical analysis is loop gain transfer function of a PL. system. As men- tioned in the introduction section, Jillusraee the hierarch of the PI..,, where focus the Frequency & Phase Detector (PFD) and the Low-Pass Filter (LPF). At Level (system level), the PLL is partitioned into its main functional blocks: PFD, LPF, Voltage-Controlled Oscillator (VCO), and Frequency Divider, and the loop gain in the phase domain can be expressed as:
L(s) K Z(s). Keno N
(5)
16

---

Published as a conference paper at ICLR 2026
Moving to Level 4, the PFD is recognized consisting of two flip-flos, AND gate, and charge pump. It acts like a phase- thase-to-aae tu-rureent gain:
Lep At(t) 2T
Lep (0rer(t) (t)) 2
K
Where, Ao(t) = (t)r(t))0p (t) the phase error, and fIt represents reference and feedback phase; To obtain the exact charge pump current Lp. we zoom into the level-3 transistor schematic of the charge pump within the PFD. At this level, lep can explicitly derived from the transistor equations as:
WNMOS
Tap R.
1 2 Mn Cox
V ov.NMOS
LNMOS
The same hierarchical analysis framework can be applied the VCO and the Frequency Divider transfer function analysis. The LPF is modeled as a simple level RLC network,
(s)
sC
This hierarchical flow results complete and quantitative framework for analyzing PLL close loop gain.
A.4 DATA COLLECTION DETAILS
Aside from the sources we mentioned in Section 2 we also have collected data from communities and online platforms dedicated to cireuit design (Learn Electronics India, AIC Design All About Circuits Embedded Wala Chegg Inc.). Each problem underwent verification by graduate students with circuit design knowledge.
To standardize the representation of problems across these diverse sources, we developed an offline Flask-based tool, Cireuit Benchmark Sample Creator (CBSC), that provides structured interface for manual data entry and organization. Using CBSC, we separately inserted problem components including circuit diagrams, difficulty levels, source information, questions, answers, and step-by- step derivations. Once the content was entered and submitted, the tool automatically generated well- well-structured folder system to store and index the problems. This workflow ensured that all benchmark entries maintained consistent formatting and organization while preserving the integrity of the original materials.
We structure the benchmark by considering folder for each question consisting of g#_question txt which have the text part the question, att_mage..pi png which is the im- age, gfltt.tt.. which save the ground-truth, @# txt which holds the multiple choice if the question requres, qf aff.a.te txt which contains the correct choice. The question folder also includes a# gddder. txt which is the step-by-step solution and c#_category. txt which is the subcate- gories for Analysis task.
A.5 EXPERIMENT MODELS DETAILS
All experiments were conducted using the following model versions and parameters:
GPT-4o: gpt-40-2024-08-06 (snapshot date: August 6, 2024)
GPT-4o-mini: openai/gpt-44-
Gemini-2.5-Prooggeee gemini cerrooopeview-eee 0605 (preview version: June 5, 2025)
Claude-Sonnet-4: claude-44 -
Intern Official release version 3.0
Qween25-V1.-72--nnsstreee Instruction-tuned version 2.5
GLM-4.5V: Vision-enabled version 4.5
Gemma-3-27B: googlee/gema- 2btit
Inference Parameters:
17

---

Published as a conference paper at ICLR 2026
Temperature: 0.1 (for all models to ensure consistency and reproducibility)
Maximum tokens: 4096
Top-p: 0.95 (where applicable)
18

---

Published as a conference paper at ICLR 2026
A.6 SYNTHETIC EXAMPLES
Synthetic Example QI: Nodal Equation
Question: Derive the nodal equation for node 2 in the s-domain. Express the equation using only the circuit elements and their values as labeled in the diagram. Make sure the final answer just the symbolic equation Vn2(s) .... where the right side contains only the labeled components and sources from the circuit diagram.
WW
R.
W
&
Corresponding Netlist:
R1
0
Ri
R6
1
2
R6
V1
2
3
Vi
R4
6
2
R4
R7
5
R7
R8
5
R88
Ground-truth:
Vn2(s) = V1 (R5 R6) (R1 + R5 + R6))
4:
Vn2(s) V3+V1
Gemini-2.5-Pro:
Vn2(s) = V1 (R5 + R6)/ + R5 + R6)
GPT-4o:
Vs(a)
Va(9)
(.(+)
V1
V/(()
R6
+
R3
R4
R6
Intern VL3-78B:
Vn2(s) = 1425///44666044444//4/4401/01113
Qwen2.5-VL-772-Inntt
Vni(s)
Vn3(s)
Vn2(s)
726
172
R3
R4
R6
R2
GLM-4.5V:
(a)
(a)(s)
Vunfa)
Van(s)
+ Vi(8)
+
Rs
R2
Rs
Rs
Ri
RS
Vn2(s)=
+
+
+ Rs
Ri
R2
Ra
R4
19

---

Published as a conference paper at ICLR 2026
Synthetie Example Q2: RLC Transfer Function
Question: Derive the nodal equation for node in the s-domain. Express the equation using only the circuit elements and their values as labeled in the diagram. Make sure the final answer is just the symbolic equation Vn3(s) where the right side contains only the labeled components and sources from the circuit diagram.
WW
L
W
10
Netlist Extraction:
11
1
4
Li
V1
1
0
step
3
R3
R4
C1
2
0
Ci
R1
2
0
R1
I2
3
2
12
R2
R5
R5
R6
2
R6
Ground-turth: X1g.0:/101SH.N..C/000.00..111211 R4*R5*s))
L1* (R1*s + R4*s + R5*5) + (R4(
Claude-Sonnet-4:
V3(s) (V6(s) *** L2 ft3 + (2()) R2 (133 (13++ (2)) ( L2* (12 R3) + 122 * R3)
Gemini-2.5-Pro:
Vn3(s) V1(s)
GPT-4o:
Va(s)
+
Vi(()
Ra
51.2
Rs
Vs(s)
8L2
VL.378B:
Vn3(s) = (V6(s) * (1/ + V2(s) * (1/R3 + 1/(s= 121) L2))
Bwen2.5-VLL-2B--mnttrtt
Vn3(s) = (Vn2(s) R2 R3)
1/(s* L2))
GLM-4.5V:
Vnn(s) sL2 n6 + R2 + R3)
2L2(Rg - Ra) + R2Rs
20

---

Published as a conference paper at ICLR 2026
Synthetic Example Q3: Small Signal (Dependent Sources)
Question: What is the transfer function from VI to R1 in this circuit?
MW
W
Netlist Extraction:
R1
2
C1
C
E1 3 2
1 2 x_1
V1 5 0 step
R2 3
R2
Ground-truth:
H(s) = * (/11* R1
R1 -1
Claude-Sonnet-4:
H(s) (R1 * + +l
Gemini-2.5-Pro:
= (
x1) + R2))
GPT-4o:
sC1
H(s) sC1
21 R₁
InterVV33-70
H(s) 1/(1 C1 R1 (1-x1))
Qwen2.55-LL-78--Instte
1 + 1/2
16
H(s) =
C1
H
GLM-4.5V:
sCiR
(1(s) sR, (1
21

---

Published as a conference paper at ICLR 2026
Synthetic Example Q5: Ideal Op-amp
Question: Derive the nodal equation for node in the s-domain. Express the equation using only the circuit elements and their values as labeled in the diagram. Make sure the final answer is just the symbolic equation Vn3(s) where the right side contains only the labeled components and sources from the circuit diagram.
Netlist Extraction:
R4
V1
step
I2
3
2
L2
C1
2
0
Ci
R1 3
Ri
6
R3 6 0 R3
Rintl 5 31 Rintl
Cintl 6 31 Cintl
Eintl 6 0 31
Ground-truth:
Vn3(s) = R1 * V (C1
L2 RA* C1 R1 RA +2 s + R1 + R4))
Gemini-2.5-Pras
Vn3(s) (R1 Vn2(s) L2 VnT(s)) s L2)
Claude-Sonnet-4
Vn3(s) = (V2(s) RI * 1.2 + V5(s) R4 * R1)/(R1
R4 + 8 L2 * (R1 R4))
GPT-4o:
Vo(s)
Vi(()
V2(3)
Vi((s
+
+
R1
8L.2
R4
R1
8T22
R4
Qwen2.5-VL-72B:
Vn3(s) Vn2(s)/(1 + R1/(s L2) R1/R3) (3)
GLM4.5V:
R
Vna(s)
(R1 + R4) + s(L2 + C1 + (C
22

---

Published as a conference paper at ICLR 2026
Synthetic Example Q6: Enhanced Control System
Question: What is the Transfer function of the provided block diagram?
R(a)
G1(s)
A0
119-21-17
G2(6)
G3(s)
C(6)
Ground-truth:
C(s)
+2*+2
R(s)
50
(2=+t))
Claude-Sonnet-4
C(s)/R(s) 10(s 2)/((* 1)3 2) +3) + 50]
Gemini-2.5-Pro:
C(s)/R(s) = (10s² + 40s 40)/(8 85 + 24s + 884s² + 223s 156)
GPT-4o:
10(s 2)
C(s)/R(s) 86 + 255 8 + 2x + 82 6s + 54
Intern VL3-78B
20(s 2
C(s)/((s)
(82 + +28 + 1)( (s + 4) + 100s(s + ()/ +6)
Qwen2.5-VL-72B:
C(s)
10(s +
R(s)
(52 + 28 + 1) + 1) + 3) + 50
GLM-4.5V:
10(s 2)
C(s)/R(s)
(s+ + () + 2) 3) + 50
23

---

Published as a conference paper at ICLR 2026
Design Example Q6: Simulation-needed Schematic-leve question
Design the sizing and biasing voltage of an Op-Amp in SKY130mm (VDD 1.8 V) shown in the provided circuit image.
Question:
VN
24

---

Published as a conference paper at ICLR 2026
A.7
CURATED PROBLEMS
Question:
This problem deals with the op amp shown the provided circuit image. All device lengths are 1 μm, the slew rate +10V/us, the GB is 10MHz, the maximum output voltage is +2V, the minimum output is -2V, and the input common mode range from -1V +2V. Design all W value of all transistors this op amp. Your design must meet or exceed the specifications. Ignore bulk effects in this problem.
Image:
+3V
Source:
M4
M14
1.51
11.51
ECE 64 6412-Spring 2003-HW8
M6
M7
1
Text Answer:
W 15 4um
Y)
MI
YouL
M8
M9
W1 W2 = 36um
W 14 16pm
100
M2
10pF
W3 W+ = W6 W 24
M13
M12
M5
MI1
Wa = W, = W = W 11 = 121um
Level:
Level
W2 W 13 W5 1.4pm
Example I: schematic-level design
Question:
Design DPLL using the tri-state topology seen in the provided circuit image that generates clock signal at frequency of 100M Hz from 50 MHz square wave input. This application of the DPLL is called frequency synthesis.
Image:
VDD
Kpomi
data in
PDni
Kf
datta
data
Up
clock JU
Ri
Viaco
VCO Kvcols
A4
Qloock
dclock Down
R2
Divider
B I/N
dolook
N Deooot
25

---

Published as a conference paper at ICLR 2026
CMOS Circuit Design, Layout, and Simulation P577
Source:
Example19.4
Level:
Level 2
Text Answer:
C =00FF, R₂ 20k0, R 42.5kΩ
Example 2: block-level design
Question:
Design a CMOS operational amplifier powered from a single 5V supply in which all MOSFET channel lengths are fixed 1um and every device operates saturation; choose the width of every transistor so that the amplifier meets or exceeds the following specifications: slew rate 110 /μs maximum and minimum output voltage Youcmmar 4V, V mon = 1V, input common-mode range VicGmane) 1.5V to VIc = 4V And unity-gain ba =10 MHz; ignore bulk/body effects. Provide a summary table (round each the nearest micron) listing the W of every transistor.
Image:
Level:
Level
Voo
ECE 6412-Spring
I
Source:
2005 HW07
Vap
M7
M1
M2
You!
Text Answer:
1/
Vetu
Js
Vav
M5
25pF
W, 90pm,
M10
W; = W W6 W, W3 40pm,
W, = W 10 W11 11 18um,
I5 250μA
Example 3: hierarchical design
Question:
In the circuit, R=20, L=1mH, and C=0.4uF. Find the resonant frequency.
Level:
Level 0.5
Source: Fundamentals of Electric Circuits
P674
Example 14.7
26

---

Published as a conference paper at ICLR 2026
Image:
Text Answer:
wy=50 krad/s
ww
Example 4: Level 0.5
Question:
For the emitter follower output stage shown below, find the value of when R
-VEE-VBE 7.826KΩ and Vcc -VEE
1Q
2.5V, VCE 0.2V, VBE 0.7V, RL 10KΩ
Image:
Level:
Level 2
Vec
fgy
+0
Q1
ECE 6412 2412-Spring
Source:
SRI
Tg]
Rr. S YOUT
2004-Homework0z
Q8
Text Answer:
23 %
VEz
Example Level 2
Question:
Use nodal analysis to determine voltages V2 in the circuit.
Image:
Level:
Level 1.5
Fundamentals of Electric Circuits
Source:
P118
13 WW
Example3.27
W/
Text Answer:
375 mV
Example 6: Level 1.5
Question:
Determine Vo in the op amp circuit
Image:
Level:
Level
wWW
Fundamentals of Electric Circuits
WW
Source:
P674
Example 5.4
1V
Text Answer:
-6v
Example Level
27

---

Published as a conference paper at ICLR 2026
A.8 PROMPT TEMPLATES
Prompt Template for Circuit Schematic Synthetic Pipeline
You are an expert electrical engineer specializing in circuit analysis. Analyze the circuit diagram and solve for the requested symbolic expression. Task: {Main Question}
Instructions: 1. Use EXACT component labels as shown in the circuit (e.g.. R1, R2, C1, C2, LI, not generic R, C, L) 2. For Laplace domain, use lowercase 's' as the complex frequency variable 3. Use standard impedances: R for resistors, I/(sC) for capacitors, sL for inductors 4. For op-amps: Apply virtual short (V+ = V-) if in negative feedback, use Ad for gain if specified
Response Format: You MUST structure your response exactly as follows:
<think>
[Show your reasoning and intermediate steps here] Identify components and nodes Inter-
mediate steps Show equations Show algebraic manipulation Any simplification steps
</think>
<answer>
[Only the final symbolic equation here, e.g., H(s) =... Vnl(s) = .... etc.] </answer>
Make sure to use standard mathematical notation with for multiplication, / for division, and for powers."
Prompt Template for Block Diagram
You are an expert in control systems engineering. Your task analyze the provided control system block diagram and derive the overall transfer function, C(s)/(ss), in its symbolic form, Question: (Main Question} Instruction: The final expression must in terms of 's'. Don't 't include the high-level symbolic block gains (e.g., Gl(s), G2(s), Hi(s), Fi(s)) in the final answer. Place your final, exact symbolic equation inside <think> tags.
<think>
C(s)/R(s) = (s + ()/ + 5s + 6)
<\think> The final answer must be rational function of 's'.
A.9 LLM USAGE
Large language models were used solely for grammar checking and minor text polishing during manuscript preparation. No LLMS were involved in research ideation, experimental design, data analysis, or substantive writing of the paper.
28


Reviews:
[{"review_id": "vIGfwweGDT", "reviewer": "Reviewer_h46k", "summary": "The paper introduces CircuitSense, a large-scale (8,006 problems across six hierarchical abstraction levels spread across Perception, Analysis, and Design problems) benchmark designed to evaluate visual mathematical reasoning in circuit analysis and design. This remains an under-explored domain for multimodal large language models (MLLMs). \n\nThe authors construct a hierarchical synthetic data generation pipeline capable of automatically producing novel circuit schematics and block diagrams with guaranteed symbolic ground-truth equations. They achieved this via SPICE simulation and symbolic analysis via SymPy and Lcapy. This allows systematic testing of models’ ability to convert visual circuit representations into symbolic transfer functions and equations, which is a step beyond conventional recognition or numerical tasks that exist in the current VLM’s. \n\nEvals across six leading MLLMs such as GPT-4o, Gemini-2.5-Pro, Claude-Sonnet-4, InternVL3, etc. reveal a sharp contrast between perception and reasoning: while models achieve >85% accuracy on component recognition and topology understanding, symbolic derivation accuracy collapses below 19%, with catastrophic failure on novel synthetic circuits. This demonstrates the baseline reliability of the benchmark in evaluating SOTA models on these tasks.", "questions": "- Beyond algebraic identity via SymPy, have you considered functionally equivalent but structurally different forms? This might influence the reported accuracy at <19%. \n\n- How closely do generated circuits reflect real textbook or industrial designs?\n\n- How did you ensure fairness when benchmarking Gemini models using Gemini as a judge?\n\n- Could CircuitSense outputs be integrated into simulation-based tools to test end-to-end utility for analog designers?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- The paper tackles an unexplored domain: hierarchical visual symbolic reasoning in circuit systems, with high-quality richly annotated data. It seems to be the first benchmark to evaluate how well AI systems connect circuit visuals to mathematical models. Prior work such as ChipVQA only partially evaluates this capability. \n- Six abstraction levels mirror real analog design processes and provide a structured way to locate where model failures occur\n- SDG pipeline and datasets are open-sourced and readily reproducible and could constitute a major contribution to the community in solving the data scarcity problem\n- Failure analysis is fine-grained, highlighting, for example, precise points of reasoning breakdown (output impedance and input impedance)", "weaknesses": "- Sim2Real gap remains: while the synthetic generation pipeline is well engineered, the circuits are constrained to 12-15 components; data sourcing from internet data might result in contamination \n- The benchmark relies on Gemini as the judge, which may result in potential biases toward the Gemini family \n- The paper could benefit in an analysis on cross-domain transfer; for example, training on schematic-level circuits and testing on block diagrams \n- Lastly, the paper would benefit in adding human baseline performance", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8bS1JM0Lmv", "reviewer": "Reviewer_bBsZ", "summary": "This paper introduces CircuitSense, a hierarchical benchmark designed to evaluate multimodal large language models on circuit understanding and symbolic reasoning tasks across over 8,000 problems. The study shows that while models perform well on visual perception tasks, they struggle significantly with symbolic derivation, highlighting a critical gap between visual comprehension and mathematical reasoning in engineering design.", "questions": "How is the correctness of the LLM’s answers evaluated? For example, in the Synthetic Example Q5, there seem to be multiple mathematically correct equations that can represent the same diagram. How is such ambiguity handled during evaluation?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The dataset is highly diverse, encompassing various difficulty levels and a wide range of analog question types. This diversity could significantly benefit future research on large language models (LLMs) in analog reasoning.\n\n- The experimental evaluation is thorough, involving multiple LLMs with different capability levels, providing a comprehensive comparison.", "weaknesses": "- Although the work is valuable, several existing benchmarks—such as AMSBench and MMCircuitEval—appear to share similar objectives and formulations. The differences between this work and those benchmarks are not clearly articulated.\n\n- In the supplementary materials, there are multiple zero-byte files (e.g., in the Perception/func directory), which may indicate missing or corrupted data.\n\nTypos:\n\nIn Table 4, “GPT-4O” should be corrected to “GPT-4o”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k3ftDdoDP3", "reviewer": "Reviewer_222z", "summary": "This paper presents CircuitSense, a comprehensive hierarchical benchmark for assessing visual-to-symbolic reasoning in circuit understanding.  It focuses on the core engineering ability to translate schematic and system diagrams into mathematical equations, which has not been tested in previous multimodal benchmarks.\nCircuitSense has 8,006 problems organized into six hierarchy levels (resistor, RLC, small signal, transistor, block, and system) and three task categories (perception, analysis, and design).  The authors propose a synthetic generation pipeline based on SPICE simulation and Mason's gain formula for block diagrams to ensure ground-truth symbolic equations.\nExperiments on six cutting-edge MLLMs (GPT-4o, Gemini-2.5-Pro, Claude-Sonnet-4, InternVL3, Qwen2.5-VL, GLM-4.5V) show that while closed-source models achieve >85% accuracy on perception tasks, their performance on symbolic derivation remains <19%, indicating a severe gap between visual parsing and mathematical reasoning.", "questions": "- Annotation Quality and Validation\nHow were the 2,986 curated problems verified?  Were multiple domain experts involved, and how did the annotators agree? Were symbolic derivations manually re-checked after SPICE/Lcapy generation to avoid mismatch errors?\n\n- Synthetic Circuit Generator\nHow does the generator handle non-linear or transistor-level behavior that symbolic solvers cannot detect? Could the authors explain how \"adaptive timeouts\" were used to balance completeness and feasibility in symbolic derivation?\n\n- Data Balance and Hierarchical Sampling\nThe design subset (157 samples) is relatively small.  Do the authors intend to scale this portion using synthetic design tasks (for example, automated topology synthesis with constraints)? Are the counts roughly equal across hierarchy levels, or are some levels (such as transistor or system) underrepresented?\n\n- Table 7 shows a significant drop in output impedance derivation (8%).  Could you provide qualitative examples that demonstrate the specific algebraic or topological misunderstanding? Do models consistently misinterpret node labeling conventions (e.g., sign or orientation errors) or fail algebraic simplification?\n\n- Robustness in Equation Comparisons\nThe symbolic comparison pipeline (SymPy plus numerical validation) is great. What is the failure rate? Were there instances where algebraically correct but simplified forms (e.g., factorized expressions) were mistakenly classified as incorrect?\n\n- Potential Extensions: Do the authors intend to include temporal behaviors, such as time-domain simulations or frequency-sweep interpretation? Could integration with RAG/agent-based method serve as future baselines?\n\nAppreciate the author's work and contributions to the community!", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Novel Benchmark Scope  \nCircuitSense combines perception, analysis, and design in a single dataset that reflects real-world circuit design workflows. The six-level hierarchy captures domain progression from physical components to system architecture, and provides a uniquely structured view of how reasoning degrades with abstraction.\n\n-Rigorous Synthetic Generation Pipeline\nThe dual-stage pipeline (schematic and block diagram) ensures electrical validity and symbolic correctness. The use of SPICE simulation, Lcapy-based symbolic derivation, and Mason's gain formula ensures physics-consistent ground truths, not simply textual labels.\n\n- Comprehensive Evaluation and Diagnostics\nTesting six leading MLLMs with curated and synthetic problems (multiple-choice -- open-ended -- symbolic derivation) reveals how pattern-matching fails when symbolic reasoning is needed.\n\n- Fine-grained Task Taxonomy \nThe benchmark explicitly distinguishes perception subtasks (component detection, connection identification, function classification) from analytical derivation, allowing for the identification of the true bottleneck: symbolic manipulation.\n\n- Strong Domain Validity \nQuestions are sourced from reputable textbooks (Gray, Razavi, Holberg) and university courses to ensure educational and professional relevance.", "weaknesses": "- Limited Novelty in Technical Methodology \nAlthough the domain is novel, the benchmark creation adheres to standard dataset synthesis paradigms (grid-based placement, random topology, template verification).  The originality is primarily in hierarchical symbolic labeling, not algorithmic innovation. Limited Novelty in \n\n-Imbalanced Data Composition \nThe dataset heavily favors analysis (approximately 7,000 samples) over design (~150 samples), making it difficult to statistically interpret design-task outcomes.\n\n- Synthetic Domain Gap \nThe significant drop in synthetic performance could be due to distribution shift rather than pure reasoning failure. The paper could better quantify the visual and algebraic difficulty of synthetic problems to address potential domain difference of the synthetic data.\n\n-Limited Evaluation Diversity \nAll tested models are general MLLMs, with no domain-specific baselines (such as SPICE-aware symbolic solvers or retrieval-augmented MLLMs) included.  This absense makes it unclear whether the failures are the result of poor general reasoning or a lack of domain knowledge.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BThCu5tyRD", "reviewer": "Reviewer_Pqrx", "summary": "This paper introduces CircuitSense, a benchmark designed to evaluate the circuit-understanding capabilities of multimodal large language models (MLLMs). CircuitSense comprises over 8,000 problems organized into a six-level hierarchical structure. The paper also presents an evaluation of six state-of-the-art MLLMs, revealing that while these models demonstrate strong visual perception, they perform catastrophically poorly in the symbolic reasoning tasks essential for circuit analysis and design.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The paper is well-written and easy to follow.\n\n2. Addressing multimodal engineering problems is an important and timely topic for current MLLMs.\n\n3. The proposed hierarchical synthetic generation pipeline is interesting.", "weaknesses": "1. Lack of discussion on related work (EEE-Bench [1]).\nEEE-Bench is a comprehensive multimodal benchmark for electrical and electronics engineering, which is highly relevant to this paper. The authors fail to acknowledge or compare their work against it.\n\n2. Overstated claims and limited novelty.\nThe paper claims to introduce the first multi-level visual-to-analytical benchmark. However, EEE-Bench has already covered comprehensive circuit analysis problems. The authors do not clearly articulate the differences between CircuitSense and EEE-Bench. In my view, the main tasks (circuit-related problems) in this paper largely overlap with those in EEE-Bench, suggesting limited novelty. Moreover, EEE-Bench addresses a broader domain of electrical and electronics engineering.\n\n3. Insufficient evaluation.\nThe paper only evaluates six large scale models, which is not enough to provide a comprehensive assessment. More open-source small models and recent proprietary models—such as o3 and o4-mini—should be included.\n\n4. Lack of new insights in the experimental results.\nBeyond the introduction of the benchmark itself, the experiments do not provide any particularly novel findings. The observation that MLLMs excel in visual perception but struggle with symbolic reasoning is already well known.\n\n5. Missing implementation details.\nThe paper does not provide sufficient information about the evaluation setup, such as the prompts used for answer extraction or accuracy calculation or other experimental configurations.\n\n6. Inconsistent complexity-level design.\nThe benchmark defines multiple complexity levels, but the experimental results do not reflect the expected performance degradation with increasing difficulty. If model performance does not decrease with complexity, it raises questions about the validity and meaning of the defined complexity levels.\n\nReference\n\n[1] EEE-Bench: A Comprehensive Multimodal Electrical and Electronics Engineering Benchmark. CVPR 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)