Architect of the Bits World: Masked Autoregressive Modeling for Circuit Generation Guided by Truth Table

## Abstract

Abstract Logic synthesis, a critical stage in electronic design automation (EDA), optimizes gate-level circuits to minimize power consumption and area occupancy in integrated circuits (ICs).
Traditional logic synthesis tools rely on human-designed heuristics, often yielding suboptimal results.
Although differentiable architecture search (DAS) has shown promise in generating circuits from truth tables, it faces challenges such as high computational complexity, convergence to local optima, and extensive hyperparameter tuning.
Consequently, we propose a novel approach integrating conditional generative models with DAS for circuit generation.
Our approach first introduces CircuitVQ, a circuit tokenizer trained based on our Circuit AutoEncoder
We then develop CircuitAR, a masked autoregressive model leveraging CircuitVQ as the tokenizer.
CircuitAR can generate preliminary circuit structures from truth tables, which guide DAS in producing functionally equivalent circuits.
Notably, we observe the scalability and emergent capability in generating complex circuit structures of our CircuitAR models.
Extensive experiments also show the superior performance of our method.
This research bridges the gap between probabilistic generative models and precise circuit generation, offering a robust solution for logic synthesis.

## 1 Introduction

With the rapid advancement of technology, the scale of integrated circuits (ICs) has expanded exponentially.
This expansion has introduced significant challenges in chip manufacturing, particularly concerning power and area metrics.
A primary objective in IC design is achieving the same circuit function with fewer transistors, thereby reducing power usage and area occupancy.

Logic synthesis [11], a critical step in electronic design automation (EDA), transforms behavioral-level circuit designs into optimized gate-level circuits, ultimately yielding the final IC layout.
The primary goal of logic synthesis is to identify the physical implementation with the fewest gates for a given circuit function.
This task constitutes a challenging NP-hard combinatorial optimization problem [12].
Current logic synthesis tools [4, 32] rely on human-designed heuristics, often leading to sub-optimal outcomes.

Differentiable architecture search (DAS) techniques [22, 8] offer novel perspectives on addressing challenges in this problem.
Circuit functions can be represented through truth tables, which map binary inputs to their corresponding outputs.
Truth tables provide a precise representation of input-output relationships, ensuring the design of functionally equivalent circuits.
Inspired by this, researchers [14, 31] have begun exploring the application of DAS to synthesize circuits directly from truth tables.
Specifically, [14] proposed CircuitNN, a framework that learns differentiable connection structures with logic gates, enabling the automatic generation of logic circuits from truth tables.
This approach significantly reduces the complexity of traditional circuit generation.
Building on this, [31] introduced T-Net, a triangle-shaped variant of CircuitNN, incorporating regularization techniques to enhance the efficiency of DAS.

Despite these advancements, several challenges remain.
The computational complexity of DAS grows quadratically with the number of gates, posing scalability issues.
Although triangle-shaped architecture [31] partially mitigates this problem, redundancy persists.
Additionally, DAS is susceptible to converging to local optima [22], where network depth and layer width require extensive searches.
The challenges arise from the vast search space in DAS.
Intuitively, limiting the search space through predefined parameters, including network depth, gates per layer, and connection probabilities, can significantly reduce the complexity.

Recent advances [24, 1, 10, 20] in conditional generative models have demonstrated remarkable performance across language, vision, and graph generation tasks.
Motivated by these developments, we propose a novel approach to circuit generation that generates preliminary circuit structures to guide DAS in generating refined circuits matching specified truth tables.
Firstly, we introduce CircuitVQ, a tokenizer with a discrete codebook for circuit tokenization.
Built upon our Circuit AutoEncoder framework [15, 18, 33], CircuitVQ is trained through a circuit reconstruction task.
Specifically, the CircuitVQ encoder encodes input circuits into discrete tokens using a learnable codebook, while the decoder reconstructs the circuit adjacency matrix based on these tokens.
Subsequently, the CircuitVQ encoder serves as a circuit tokenizer for CircuitAR pretraining, which employs a masked autoregressive modeling paradigm [7, 19].
In this process, the discrete codes function as supervision signals.
After training, CircuitAR can generate discrete tokens progressively, which can be decoded into initial circuit structures by the decoder of CircuitVQ.
These prior insights can guide DAS in producing refined circuits that match the target truth tables precisely.
Our key contributions can be summarized as follows:

- •
We introduce CircuitVQ, a circuit tokenizer that facilitates graph autoregressive modeling for circuit generation, based on our Circuit AutoEncoder framework;
- •
Develop CircuitAR, a model trained using masked autoregressive modeling, which generates initial circuit structures conditioned on given truth tables;
- •
Propose a refinement framework that integrates differentiable architecture search to produce functionally equivalent circuits guided by target truth tables;
- •
Comprehensive experiments demonstrating the scalability and capability emergence of our CircuitAR and the superior performance of the proposed circuit generation approach.

## 2 Preliminaries

### 2.1 Modeling Circuit as DAG

In this work, we model the circuit as a directed acyclic graph (DAG) [5], which facilitates graph autoregressive modeling.
Specifically, each node in the DAG corresponds to a logic gate, while the directed edges represent the connections between these components.

### 2.2 Differentiable CircuitNN

As depicted in [Figure 1](https://arxiv.org/html/2502.12751v2#S2.F1), CircuitNN [14] replaces traditional neural network layers with logic gates (e.g., NAND) as basic computational units, learning to synthesize circuits by optimizing logic correctness based on truth tables.
During training, input connections of each gate are determined through learnable probability distributions, enabling adaptive circuit architecture modification.
To enable gradient-based learning, CircuitNN transforms discrete logic operations into continuous, differentiable functions using NAND gates for simplicity.
The NAND gate is logically complete, allowing the construction of any complex logic circuit.
Its continuous relaxation can be defined as:

$$ $\text{NAND}(x,y)=1-x\cdot y,\text{ where }x,y\in[0,1].$ (1) $$

Additionally, CircuitNN employs Gumbel-Softmax [16] for stochastic sampling of gate inputs.
Through stochastic relaxation, gate and network outputs are no longer binary but take continuous values ranging from 0 to 1 instead.
This end-to-end differentiability allows the model to learn gate input distributions using gradient descent.
After training, the continuous, probabilistic circuit is converted back into a discrete logic circuit by selecting the most probable connections based on the learned probability distributions, as shown in [Figure 1](https://arxiv.org/html/2502.12751v2#S2.F1).

Figure: Figure 1: Illustration of differentiable CircuitNN.
Refer to caption: x1.png

## 3 Methodology

In this section, we first introduce CircuitVQ ([Section 3.2](https://arxiv.org/html/2502.12751v2#S3.SS2)), a model built upon the Circuit AutoEncoder framework ([Section 3.1](https://arxiv.org/html/2502.12751v2#S3.SS1)) and trained with the task of circuit reconstruction.
Utilizing CircuitVQ as a tokenizer, we subsequently train CircuitAR ([Section 3.3](https://arxiv.org/html/2502.12751v2#S3.SS3)) with graph autoregressive modeling paradigm, which can generate preliminary circuit structures conditioned on a provided truth table.
Finally, the initial circuit structure generated by CircuitAR serves as a guide for DAS ([Section 3.4](https://arxiv.org/html/2502.12751v2#S3.SS4)) to refine and generate circuits functionally equivalent to the given truth table.

### 3.1 Circuit AutoEncoder

Let $\mathcal{G}=(\mathcal{V},\mathcal{A})$ represent a circuit, where $\mathcal{V}$ denotes the set of $N$ nodes, with each node $v_{i}\in\mathcal{V}$.
Following the architecture of CircuitNN [14, 31], each node $v_{i}$ can be classified into one of three types: primary inputs (PIs), primary outputs (POs), and NAND gates, each labeled by $u_{i}\in\mathcal{U},i\in\{1,2,3\}$ respectively.
The adjacency matrix $\mathcal{A}\in\{0,1\}^{N\times N}$ captures the connectivity between nodes, where $\mathcal{A}_{i,j}=1$ indicates the presence of a directed edge from $v_{i}$ to $v_{j}$.

In the circuit autoencoder framework, an encoder, denoted as $g_{E}$, encodes the circuit $\mathcal{G}$ into a latent representation $\bm{Z}\in\mathbb{R}^{N\times d}$ with dimensionality $d$.
The encoding process for a circuit can be formulated as:

$$ $\bm{Z}=g_{E}(\mathcal{V},\mathcal{A}).$ (2) $$

Simultaneously, a decoder $g_{D}$ aims to reconstruct the original circuit $\mathcal{G}$ from the latent representation $\bm{Z}$.
Since node types can be directly derived from the truth table, the decoder is designed to focus on reconstructing the adjacency matrix $\mathcal{A}$, which can be formalized as follows:

$$ $\displaystyle\tilde{\mathcal{G}}=(\mathcal{V},\tilde{\mathcal{A}})=(\mathcal{V },f(g_{D}(\bm{Z},\mathcal{V}))),$ (3) $$

where $\tilde{\mathcal{A}}\in\mathbb{R}^{N\times N}$ denotes the reconstructed adjacency matrix, obtained by decoding $\bm{Z}$ through $g_{D}$ and applying a mapping function $f:\mathbb{R}^{N\times d}\rightarrow\mathbb{R}^{N\times N}$.
Meanwhile, $\tilde{\mathcal{G}}$ represents the reconstructed graph.
A robust encoder $g_{E}$ capable of capturing fine-grained structural information is essential to facilitate the circuit reconstruction task.
We incorporate the Graphormer [34] architecture into $g_{E}$.
For the decoder $g_{D}$, we adopt a simple Transformer-based [9] architecture, as an overly powerful decoder could negatively impact the performance of the circuit tokenizer.

### 3.2 CircuitVQ

As mentioned in [Section 3.1](https://arxiv.org/html/2502.12751v2#S3.SS1), we propose a circuit autoencoder architecture for the circuit reconstruction task.
The outputs of $g_{E}$ and the inputs of $g_{D}$ are continuous.
The circuit tokenizer is required to map the circuit to a sequence of discrete circuit tokens for masked autoregressive modeling, illustrated in [Section 3.3](https://arxiv.org/html/2502.12751v2#S3.SS3).
Specifically, a circuit $\mathcal{G}$ can be tokenized to $\bm{Y}=[y_{1},y_{2},\cdots,y_{N}]\in\mathbb{R}^{N}$ using the circuit quantizer $\mathcal{C}$ which contains $K$ discrete codebook embeddings.
Here, each token $y_{i}$ belongs to the vocabulary set $\{1,2,\dots,K\}$ of $\mathcal{C}$.
Consequently, we develop a circuit tokenizer, CircuitVQ, based on the circuit autoencoder by integrating a circuit quantizer $\mathcal{C}$.
As shown in [Figure 2](https://arxiv.org/html/2502.12751v2#S3.F2), the tokenizer comprises three components: a circuit encoder $g_{E}$, a circuit quantizer $\mathcal{C}$, and a circuit decoder $g_{D}$.

Figure: Figure 2: The training process of CircuitVQ.
Refer to caption: x2.png

Firstly, $g_{E}$ encodes the circuit into vector representations $\bm{Z}$.
Subsequently, $\mathcal{C}$ identifies the nearest neighbor in the codebook for $\bm{z}_{i}\in\bm{Z}$.
Let $\{\bm{e}_{1},\bm{e}_{2},\dots,\bm{e}_{K}\}$ represent the codebook embeddings and $\bm{e}_{K}\in\mathbb{R}^{d}$.
For the $i$-th node, the quantized code $y_{i}$ is determined by:

$$ $\displaystyle y_{i}=\arg\min_{j}||\ell_{2}(\bm{z}_{i})-\ell_{2}(\bm{e}_{j})||_ {2},$ (4) $$

Figure: Figure 3: The training process of CircuitAR under the condition of the truth table, leveraging CircuitVQ as the tokenizer.
Refer to caption: x3.png

where $j\in\{1,2,\dots,K\}$ and $\ell_{2}$ normalization is applied during the codebook lookup [30, 35].
This distance metric is equivalent to selecting codes based on cosine similarity.
Consequently, the output of $\mathcal{C}$ for each node representation $\bm{z}_{i}$ the can be calculated based on the given [Equation 4](https://arxiv.org/html/2502.12751v2#S3.E4):

$$ $\displaystyle\tilde{\bm{z}}_{i}$ $\displaystyle=\mathcal{C}(\bm{z}_{i})=\ell_{2}(\bm{e}_{y_{i}}),\text{ where } \tilde{\bm{z}}_{i}\in\tilde{\bm{Z}}.$ (5) $$

After quantizing the circuit into discrete tokens, the $\ell_{2}$-normalized codebook embeddings $\tilde{\bm{Z}}=\{\tilde{\bm{z}}_{i}\}_{i=1}^{N}$ are fed to $g_{D}$.
The output vectors $\tilde{\bm{X}}=\{\tilde{\bm{x}}_{i}\}_{i=1}^{N}=g_{D}(\tilde{\bm{Z}},\mathcal{
V})$ are used to reconstruct the original adjacency matrix $\mathcal{A}$ of the circuit $\mathcal{G}$.
Specifically, the reconstructed adjacency matrix $\tilde{\mathcal{A}}$ is derived from the output vectors $\tilde{\bm{X}}$ as follows:

$$ $\displaystyle\tilde{\mathcal{A}}=f(\tilde{\bm{X}})=\sigma\left(f_{1}(\tilde{ \bm{X}})\cdot f_{2}(\tilde{\bm{X}})^{\top}\right),$ (6) $$

where both $f_{1}:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ and $f_{2}:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ are learnable projection functions, and $\sigma(x)$ denotes the sigmoid function.
The training objective of the circuit reconstruction task is to minimize the binary cross-entropy loss between the reconstructed adjacency matrix $\tilde{\mathcal{A}}$ and the original adjacency matrix $\mathcal{A}$, which can be calculated as follows:

$$ $\displaystyle\mathcal{L}_{\text{rec}}=-\frac{1}{N^{2}}\sum_{i=1}^{N}\sum_{j=1} ^{N}\bigl{[}$ $\displaystyle\mathcal{A}_{ij}\log(\tilde{\mathcal{A}}_{ij})+(1-\mathcal{A}_{ij })\log(1-\tilde{\mathcal{A}}_{ij})\bigr{]}.$ (7) $$

Given that the quantization process in [Equation 4](https://arxiv.org/html/2502.12751v2#S3.E4) is non-differentiable, gradients are directly copied from the decoder input to the encoder output during backpropagation, which enables the encoder to receive gradient updates.
Intuitively, while the quantizer selects the nearest codebook embedding for each encoder output, the gradients of the codebook embeddings provide meaningful optimization directions for the encoder.
Consequently, the overall training loss for CircuitVQ is defined as:

$$ $\displaystyle\mathcal{L}_{\text{vq}}=\mathcal{L}_{\text{rec}}+\|\bm{Z}-\text{ sg}[\bm{E}]\|_{2}^{2}+\beta\cdot\|\text{sg}[\bm{Z}]-\bm{E}\|_{2}^{2},$ (8) $$

where $\mathrm{sg}[\cdot]$ stands for the stop-gradient operator, which is an identity at the forward pass while having zero gradients during the backward pass.
$\bm{E}=\{\bm{e}_{y_{i}}\}^{N}_{i=1}$ and $\beta$ denotes the hyperparameter for commitment loss [30].

Codebook utilization.
A common issue in vector quantization training is codebook collapse, where only a small proportion of codes are actively utilized.
To mitigate this problem, empirical strategies [35, 16] are employed.
Specifically, we compute the $\ell_{2}$-normalized distance to identify the nearest code while reducing the dimensionality of the codebook embedding space [35].
These low-dimensional codebook embeddings are subsequently mapped back to a higher-dimensional space before being passed to the decoder.
Furthermore, we leverage the Gumbel-Softmax trick [16] to enable smoother token selection during training, ensuring that a broader range of tokens in the codebook are actively trained, thereby improving the overall utilization of the codebook.

Figure: Algorithm 1 Autoregressive Decoding

### 3.3 CircuitAR

After completing the CircuitVQ training, we train CircuitAR using a graph autoregressive modeling paradigm as shown in [Figure 3](https://arxiv.org/html/2502.12751v2#S3.F3), where CircuitVQ functions as the tokenizer.
Let $\bm{Y}=[y_{i}]_{i=1}^{N}$ represent the discrete latent tokens of the input circuit $\mathcal{G}$, tokenized by CircuitVQ.
During the masked autoregressive training process, we sample a subset of nodes $\mathcal{V}_{s}\subset\mathcal{V}$ and replace them with a special mask token $\mathit{m}$.
For the masked $\bm{Y}$, the latent token $\bar{y}_{i}$ is defined as:

$$ $\bar{y}_{i}=\begin{cases}y_{i},&\text{if }v_{i}\notin\mathcal{V}_{s};\\ \mathit{m},&\text{if }v_{i}\in\mathcal{V}_{s}.\end{cases}$ (9) $$

Following [7] and [20], we employ a cosine mask scheduling function $\gamma(r)=\cos(0.5\pi r)$ in the sampling process.
This involves uniformly sampling a ratio $r$ from the interval $[0,1]$ and then selecting $\lceil\gamma(r)\cdot N\rceil$ tokens from $\bm{Y}$ to mask uniformly.
Let $\bar{\bm{Y}}=[\bar{y}_{i}]_{i=1}^{N}$ denote the output after applying the masking operation to $\bm{Y}$.
The masked sequence $\bar{\bm{Y}}$ is then fed into a multi-layer transformer with bidirectional attention to predict the probabilities $p(y_{i}|\bar{\bm{Y}},\mathcal{T})$ for each $v_{i}\in\mathcal{V}_{s}$ under the condition of the truth table.
The transformer is designed based on Llama models, each CircuitAR layer consists of a self-attention block, a cross-attention block, and an FFN block.
Specifically, the info of the truth table is conditioned by serving $\mathcal{T}$ as the input key and value of the cross-attention block.
The training loss for CircuitAR is defined as:

$$ $\mathcal{L}_{\text{ar}}=-\sum\limits_{\mathcal{D}}\sum\limits_{v_{i}\in \mathcal{V}_{s}}\log p(y_{i}|\bar{\bm{Y}},\mathcal{T}),$ (10) $$

where $\mathcal{D}$ represents the set of training circuits.

Autoregressive decoding.
We introduce a parallel decoding method, where tokens are generated in parallel.
This approach is feasible due to the bidirectional self-attention mechanism of CircuitAR.
At inference time, we begin with a blank canvas $\bar{\bm{Y}}=[\mathit{m}]^{N}$ and the decoding process of CircuitAR follows [Algorithm 1](https://arxiv.org/html/2502.12751v2#alg1).
Specifically, the decoding algorithm generates a circuit in $T$ steps.
At each iteration, the model predicts all tokens simultaneously but retains only the most confident predictions following the cosine schedule [7, 20].
The remaining tokens are masked and re-predicted in the next iteration.
The mask ratio decreases progressively until all tokens are generated within $T$ iterations.

### 3.4 Differentiable Architecture Search

After completing the training process of CircuitAR, autoregressive decoding is performed based on the input truth table $\mathcal{T}$ to generate preliminary circuit structures represented by the reconstructed adjacency matrix $\tilde{\mathcal{A}}$.
This matrix $\tilde{\mathcal{A}}$ can serve as prior knowledge for DAS, enabling the generation of a precise circuit that is logically equivalent to $\mathcal{T}$.

DAG Search.
The reconstructed adjacency matrix $\tilde{\mathcal{A}}$ is a probability matrix that denotes the probabilities of connections between gates.
However, $\tilde{\mathcal{A}}$ may contain cycles and identifying the optimal DAG with the highest edge probabilities is an NP-hard problem.
Consequently, we employ a greedy algorithm to obtain a suboptimal DAG.
As illustrated in [Algorithm 2](https://arxiv.org/html/2502.12751v2#alg2), the algorithm initializes $\bar{\mathcal{A}}\in\mathbb{R}^{N\times N}$ with edge probabilities and enforces basic structural rules:
PIs have no indegree, POs have no outdegree, and self-loops are prohibited in circuit designs.
Following this initialization, a depth-first search (DFS) is conducted to detect cycles in $\bar{\mathcal{A}}$.
If no cycles are found, $\bar{\mathcal{A}}$ is a valid DAG, and the algorithm terminates.
If a cycle is detected, the edge with the lowest probability within the cycle is identified and removed by setting the corresponding edge in $\bar{\mathcal{A}}$ to 0.
This process repeats iteratively until no cycles remain.
This greedy approach ensures the derivation of a valid DAG $\bar{\mathcal{A}}$ that approximates the optimal structure while preserving the acyclic property necessary for circuit design.
The resulting DAG serves as a foundation for further refinement in the DAS process, ultimately generating a precise circuit that is logically equivalent to $\mathcal{T}$.

Initialization.
After executing [Algorithm 2](https://arxiv.org/html/2502.12751v2#alg2), the adjacency matrix of a valid DAG $\bar{\mathcal{A}}\in\mathbb{R}^{N\times N}$ and its corresponding probability matrix $\hat{\mathcal{A}}=\bar{\mathcal{A}}\cdot\tilde{\mathcal{A}}$, where $\hat{\mathcal{A}}\in\mathbb{R}^{N\times N}$, are obtained.
Using $\bar{\mathcal{A}}$, we derive the hierarchical structure $H=\{h_{1},h_{2},\dots,h_{l}\}$, where $h_{l}$ represents the node list of the $l$-th layer.
The set $H$ encapsulates the layer count $l$ and the width information of each layer, which is used to initialize CircuitNN illustrated in [Figure 1](https://arxiv.org/html/2502.12751v2#S2.F1).
For connection probabilities, since each node can only connect to nodes from preceding layers, we normalize the connection probabilities such that their summation equals 1.
This yields the weights $\bm{w}\in\mathbb{R}^{N_{p}}$ for possible connections, where $N_{p}$ denotes the number of nodes in the previous layer.
To ensure compatibility with the Softmax function applied in CircuitNN, we initialize the logits $\hat{\bm{w}}\in\mathbb{R}^{N_{p}}$ such that the Softmax output matches the normalized connection probabilities.
The logits are initialized as follows:

$$ $\hat{\bm{w}}=\log(\bm{w}+\epsilon)-\frac{1}{N_{p}}\sum_{i=1}^{N_{p}}\log(\bm{w }_{i}+\epsilon),$ (11) $$

where $\epsilon$ is a small constant for numerical stability.
After initialization, the precise circuit structure is obtained through DAS, guided by the input truth table.
Notably, if DAS converges to a local optimum, the weights of the least initialized nodes can be randomly selected and reinitialized using [Equation 11](https://arxiv.org/html/2502.12751v2#S3.E11) to facilitate further optimization.

### 3.5 Bits Distance

DAS introduces inherent randomness, complicating the evaluation of CircuitAR’s circuit generation capability using post-DAS metrics.
To overcome this, we introduce Bits Distance (BitsD), a metric offering a more reliable assessment of CircuitAR’s conditional generation ability.
BitsD quantifies the discrepancy between the outputs of an untrained CircuitNN, initialized via CircuitAR, and the labels from the truth table.
It measures how well CircuitAR generates circuits conditioned on the truth table.
Specifically, after initializing CircuitNN, we feed it with the truth table inputs and compute the mean absolute error (MAE) between the untrained CircuitNN outputs and the truth table labels.
This MAE is defined as Bits Distance.
A smaller BitsD indicates that the untrained CircuitNN is closer to the target circuit described by the truth table.

## 4 Experiments

### 4.1 Experiment Settings

Data Augmentation.
We provide more details about data augmentation in [Appendix D](https://arxiv.org/html/2502.12751v2#A4) and investigate the impact of the idle of NAND gates in [Appendix E](https://arxiv.org/html/2502.12751v2#A5).

Training Details.
We generate a training dataset with around 400k circuits (average 200 gates per circuit) from the open-source datasets [6, 2, 3].
The training dataset construction details will be illustrated in [Appendix C](https://arxiv.org/html/2502.12751v2#A3).
We also provide more details about the training processes of CircuitVQ and CircuitAR in [Section B.1](https://arxiv.org/html/2502.12751v2#A2.SS1) and [Section B.2](https://arxiv.org/html/2502.12751v2#A2.SS2).

Baseline Selection.
For baseline selection, we choose CircuitNN [14] and T-Net [31] due to their state-of-the-art (SOTA) performance in circuit generation guided by truth tables.
Additionally, several other studies [28, 21, 36] have explored circuit generation using different paradigms.
We discuss these approaches in [Appendix A](https://arxiv.org/html/2502.12751v2#A1), as they diverge from the DAS paradigm employed in this work.

Evaluation Details.
To validate the effectiveness of our CircuitAR models, we conduct evaluations using circuits from the IWLS competition [23], which include five distinct function categories: random, basic functions, Espresso [25], arithmetic functions, and LogicNets [29].
Random circuits consist of random and decomposable Boolean functions, basic functions include majority functions and binary sorters, and arithmetic functions involve arithmetic circuits with permuted inputs and dropped outputs.
Furthermore, we evaluate the BitsD for CircuitAR models with different sizes to assess their conditional circuit generation capability.
This evaluation is performed on our circuit generation benchmark with 1000 circuits separate from the training dataset.

Figure: Figure 4: Scaling behavior of CircuitAR with different model parameters.
Refer to caption: x4.png

### 4.2 Scalability and Emergent Capability

To analyze CircuitAR’s scaling behavior, we perform experiments along two primary dimensions: parameter scaling ([Figure 7](https://arxiv.org/html/2502.12751v2#S4.F7)) and data scaling ([Figure 7](https://arxiv.org/html/2502.12751v2#S4.F7)).
Our results reveal distinct performance patterns quantified through BitsD, demonstrating how these scaling axes influence performance.
Additionally, we observe emergent capability in generating complex circuit structures of CircuitAR.

Parameter Scaling.
As illustrated in [Figure 7](https://arxiv.org/html/2502.12751v2#S4.F7), increasing model capacity exhibits robust scaling laws.
The 300M parameter model achieves 0.5317 BitsD, while scaling to 2B parameters yields 0.4362 BitsD.
This progression follows a power-law relationship [17], where performance scales predictably with model size.
Notably, marginal returns diminish at larger scales.
The 0.3B$\rightarrow$0.6B transition provides a 7.94% improvement versus 6.07% for 1B$\rightarrow$2B, highlighting practical trade-offs between capacity and computational costs.
These findings corroborate theoretical expectations [26], confirming that larger models compress logical information more efficiently.
Moreover, as shown in [Figure 7](https://arxiv.org/html/2502.12751v2#S4.F7), larger models achieve better performance under a fixed compute budget despite training on fewer tokens, owing to their superior capacity.
[Figure 7](https://arxiv.org/html/2502.12751v2#S4.F7) also illustrates that BitsD scales inversely with the computing budget, which aligns with the scaling law [17] during LLM training.

Data Scaling.
[Figure 7](https://arxiv.org/html/2502.12751v2#S4.F7) illustrates consistent performance gains with increased training tokens across all sizes.
For the 2B model, BitsD improves by 8.13%, 0.4748$\rightarrow$0.4362, when scaling from 1B to 4B tokens.
Moreover, larger models exhibit superior data efficiency.
Specifically, the 2B model achieves better performance with 4B tokens than the 1B model, emphasizing the interplay between model capacity and training scale.

Emergent Capability.
[Figure 7](https://arxiv.org/html/2502.12751v2#S4.F7) highlights CircuitAR’s emergent capability in generating complex circuit structures.
A clear phase transition is observed at the 2B parameter threshold, where circuit depth increases significantly compared to the 1B model, indicating an emergent capacity for handling structural complexity. Moreover, an inverse correlation between model scale and NAND gate count reveals an efficiency paradigm.
While models ranging from 300M to 1B parameters maintain similar component counts, the 2B model achieves a reduction in NAND gates despite its increased depth, suggesting enhanced topological optimization capabilities at scale.
This emergent behavior demonstrates that increasing model parameters can enhance structural efficiency in circuit generation.

**Table 1: Experiment results of circuit generation accuracy and DAS steps. Impr. is the percentage decrease in DAS steps.**
| Benchmark | CircuitNN | T-Net | CircuitAR-2B |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Category | IWLS | # PI | # PO | Acc.(%)$\uparrow$ | Steps$\downarrow$ | Acc.(%)$\uparrow$ | Steps$\downarrow$ | Impr.(%)$\uparrow$ | Acc.(%)$\uparrow$ | Steps$\downarrow$ | Impr.(%)$\uparrow$ |
| Random | ex00 | 6 | 1 | 100 | 88715 | 100 | 85814 | 3.27 | 100 | 52023 | 41.36 |
| ex01 | 6 | 1 | 100 | 64617 | 100 | 68686 | -6.30 | 100 | 29636 | 54.14 |  |
| Basic<br>Functions | ex11 | 7 | 1 | 100 | 104529 | 100 | 49354 | 52.78 | 100 | 47231 | 54.81 |
| ex16 | 5 | 5 | 100 | 115150 | 100 | 121108 | -5.17 | 100 | 45434 | 60.54 |  |
| ex17 | 6 | 6 | 100 | 90584 | 100 | 57875 | 36.11 | 100 | 58548 | 35.66 |  |
| Expresso | ex38 | 8 | 7 | 100 | 86727 | 100 | 86105 | 0.71 | 100 | 74847 | 13.70 |
| ex46 | 5 | 8 | 100 | 75726 | 100 | 75603 | 0.16 | 100 | 26854 | 64.54 |  |
| Arithmetic<br>Function | ex50 | 8 | 2 | 100 | 87954 | 100 | 65689 | 25.31 | 100 | 42729 | 51.42 |
| ex53 | 8 | 2 | 100 | 92365 | 100 | 75140 | 18.65 | 100 | 68246 | 38.26 |  |
| LogicNet | ex92 | 10 | 3 | 100 | 220936 | 100 | 206941 | 6.33 | 100 | 134192 | 39.26 |
| Average | 100 | 102730 | 100 | 88831 | 13.19 | 100 | 57974 | 45.37 |  |  |  |

**Table 2: Experiment results of circuit generation size. Impr. represents the percentage decrease in search space and used NAND gates.**
| Benchmark | CircuitNN | T-Net | CircuitAR-2B |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Category | IWLS | # NAND$\downarrow$ | # NAND$\downarrow$ | Impr.(%)$\uparrow$ | # NAND$\downarrow$ | Impr.(%)$\uparrow$ |  |  |  |  |  |
| Search Space | Used | Search Space | Used | Search Space | Used | Search Space | Used | Search Space | Used |  |  |
| Random | ex00 | 700 | 58 | 400 | 68 | 42.86 | -17.24 | 126 | 61 | 82.00 | -5.17 |
| ex01 | 700 | 66 | 400 | 62 | 42.86 | 6.06 | 138 | 66 | 80.29 | 0.00 |  |
| Basic<br>Functions | ex11 | 300 | 52 | 180 | 52 | 40.00 | 0.00 | 98 | 45 | 67.33 | 13.46 |
| ex16 | 700 | 78 | 400 | 59 | 42.86 | 24.36 | 113 | 57 | 83.86 | 26.92 |  |
| ex17 | 800 | 109 | 500 | 98 | 37.50 | 10.09 | 196 | 95 | 75.50 | 12.84 |  |
| Expresso | ex38 | 800 | 98 | 500 | 94 | 37.50 | 4.08 | 178 | 86 | 77.75 | 12.24 |
| ex46 | 800 | 77 | 500 | 78 | 37.50 | -1.30 | 161 | 79 | 79.88 | -2.60 |  |
| Arithmetic<br>Functions | ex50 | 300 | 59 | 180 | 56 | 40.00 | 5.09 | 77 | 48 | 74.33 | 18.64 |
| ex53 | 1000 | 118 | 600 | 116 | 40.00 | 1.70 | 185 | 111 | 81.50 | 5.93 |  |
| LogicNet | ex92 | 1000 | 99 | 600 | 90 | 40.00 | 9.09 | 168 | 86 | 83.20 | 13.13 |
| Average | 710 | 81.40 | 426 | 77.30 | 40.11 | 4.19 | 144 | 73.40 | 78.56 | 9.54 |  |

### 4.3 SOTA Circuit Generation

Given the superior performance of CircuitAR-2B, as demonstrated in [Section 4.2](https://arxiv.org/html/2502.12751v2#S4.SS2), we employ it to generate preliminary circuit structures conditioned on truth tables, which are subsequently refined using DAS.
Detailed experimental results are presented in [Table 1](https://arxiv.org/html/2502.12751v2#S4.T1) and [Table 2](https://arxiv.org/html/2502.12751v2#S4.T2).

Efficiency.
As illustrated in [Table 1](https://arxiv.org/html/2502.12751v2#S4.T1), CircuitAR-2B achieves a 45.37% average improvement in optimization steps compared to CircuitNN, while maintaining 100% accuracy according to the provided truth tables.
This performance significantly surpasses T-Net’s 13.19% improvement.
The substantial reduction in optimization steps indicates that the preliminary circuit structures generated by CircuitAR-2B effectively prune the search space without compromising the quality of DAS.

Effectiveness.
[Table 2](https://arxiv.org/html/2502.12751v2#S4.T2) demonstrates that CircuitAR-2B reduces NAND gate usage by an average of 9.54% compared to CircuitNN, while simultaneously reducing the search space by 78.56%.
Notably, for both basic functions (e.g., ex16, with a 26.92% reduction) and complex benchmarks (e.g., ex92, with a 13.13% reduction), our method exhibits superior hardware resource utilization compared to the baseline approaches.
This dual improvement in search space compression and circuit compactness underscores the effectiveness of the preliminary circuit structures generated by our CircuitAR-2B under the condition of the truth tables.

Critically, the 100% accuracy across all benchmarks confirms that our method maintains functional correctness while achieving these efficiency gains.
This is guaranteed by the DAS process.
Specifically, the training does not terminate until the loss converges to a near-zero threshold.
At this point, the generated circuit is functionally equivalent to the target truth table, ensuring perfect accuracy.
This is not merely an empirical observation but a direct result of the rigorous optimization process in DAS, which enforces logical correctness by design.
These results validate our hypothesis that integrating learned structural priors with CircuitAR enables more sample-efficient circuit generation compared to basic CircuitNN [14] and template-driven [31] DAS approaches.

Circuit Size.
Compared to prior probabilistic generative models [21, 28, 36], our method achieves an order-of-magnitude improvement in directly generatable circuit scale, which is a significant advance, especially given the exponential complexity growth typical in the scaling of circuit size.
In practical circuit optimization [13], large circuits are typically partitioned into smaller subcircuits for tractable optimization.
Our primary focus is to explore the direct capabilities of generative models in circuit generation.
Current evaluation allows us to evaluate the core contributions of our approach without focusing on the additional complexity of decomposition and reintegration.

### 4.4 Ablation Study

Figure: Figure 8: Ablation study on the initialization with edge probability generated by CircuitAR-2B.

We conducted an ablation study to evaluate the effectiveness of the probability matrix $\hat{\mathcal{A}}$ generated by CircuitAR-2B.
As summarized in [Figure 8](https://arxiv.org/html/2502.12751v2#S4.F8), the experiment results reveal that both variants achieve 100% accuracy across all benchmarks, suggesting that the initialization process does not impair the ability to generate functionally equivalent circuits.
The primary distinction lies in the efficiency of the search process, quantified by the number of search steps.
[Figure 8](https://arxiv.org/html/2502.12751v2#S4.F8) underscores the significance of the initialization process, demonstrating that our CircuitAR models can produce high-quality preliminary circuit structures, which can guide the subsequent DAS process effectively.

## 5 Conclusion

In this paper, we propose a novel approach that integrates conditional generative models with DAS for circuit generation.
Our framework begins with the design of CircuitVQ, a circuit tokenizer trained using a Circuit AutoEncoder.
Building on this, we develop CircuitAR, a masked autoregressive model that utilizes CircuitVQ as its tokenizer.
CircuitAR can generate preliminary circuit structures directly from truth tables, which are then refined by DAS to produce functionally equivalent circuits.
The scalability and capability emergence of CircuitAR highlights the potential of masked autoregressive modeling for circuit generation tasks, akin to the success of large models in language and vision domains.
Extensive experiments demonstrate the superior performance of our method, underscoring its efficiency and effectiveness.
This work bridges the gap between probabilistic generative models and precise circuit generation, offering a robust and automated solution for logic synthesis.

## 6 Limitations

This study presents a preliminary investigation of scaling laws under current computational constraints.
Due to limited computing resources, the research is intentionally bound to models operating within restricted model parameters and training data.
Although our experimental framework demonstrates a tenfold increase in circuit complexity compared to prior works [21, 36, 28], there is substantial potential for further improvement in circuit scale.

## References

- [1]
J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, et al.
Accurate Structure Prediction of Biomolecular Interactions with AlphaFold 3.
Nature, pages 1–3, 2024.
- [2]
C. Albrecht.
IWLS 2005 Benchmarks.
In IEEE/ACM International Workshop on Logic Synthesis, 2005.
- [3]
L. Amarú, P.-E. Gaillardon, and G. De Micheli.
The EPFL Combinational Benchmark Suite.
In IEEE/ACM International Workshop on Logic Synthesis, 2015.
- [4]
R. Brayton and A. Mishchenko.
ABC: An Academic Industrial-Strength Verification Tool.
In Computer Aided Verification: 22nd International Conference, CAV 2010, Edinburgh, UK, July 15-19, 2010. Proceedings 22, pages 24–40. Springer, 2010.
- [5]
R. Brummayer and A. Biere.
Local Two-Level and-Inverter Graph Minimization Without Blowup.
Proc. MEMICS, 6:32–38, 2006.
- [6]
D. Bryan.
The ISCAS’85 Benchmark Circuits and Netlist Format.
North Carolina State University, 1985.
- [7]
H. Chang, H. Zhang, L. Jiang, C. Liu, and W. T. Freeman.
MaskGIT: Masked generative image transformer.
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- [8]
X. Chu, X. Wang, B. Zhang, S. Lu, X. Wei, and J. Yan.
DARTS-: Robustly stepping out of performance collapse without indicators.
arXiv preprint arXiv:2009.01027, 2020.
- [9]
A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al.
The llama 3 herd of models.
arXiv preprint arXiv:2407.21783, 2024.
- [10]
P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, F. Boesel, et al.
Scaling Rectified Flow Transformers for High-Resolution Omage Synthesis.
URL https://arxiv. org/abs/2403.03206, 2, 2024.
- [11]
G. D. Hachtel and F. Somenzi.
Logic Synthesis and Verification Algorithms.
Springer Science & Business Media, 2005.
- [12]
G. D. Hachtel and F. Somenzi.
Logic synthesis and verification algorithms.
Springer Science & Business Media, 2005.
- [13]
A. Hillier et al.
Learning to design efficient logic circuits.
IWLS, 2023.
- [14]
A. Hillier, N. N. Vũ, D. J. Mankowitz, D. Calandriello, E. Leurent, G. Rotival, I. Lobov, K. Mahajan, M. Gelmi, and N. Antropova.
Learning to Design Efficient Logic Circuits.
2023.
https://cassyni.com/events/S2LPTWZeMh9TGcLJe5jpqK.
- [15]
Z. Hou, X. Liu, Y. Cen, Y. Dong, H. Yang, C. Wang, and J. Tang.
GraphMAE: Self-Supervised Masked Graph AutoEncoders.
In ACM International Conference on Knowledge Discovery and Data Mining (KDD), pages 594–604, 2022.
- [16]
E. Jang, S. Gu, and B. Poole.
Categorical Reparameterization with Gumbel-Softmax.
arXiv preprint arXiv:1611.01144, 2016.
- [17]
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei.
Scaling laws for neural language models.
arXiv preprint arXiv:2001.08361, 2020.
- [18]
J. Li, R. Wu, W. Sun, L. Chen, S. Tian, L. Zhu, C. Meng, Z. Zheng, and W. Wang.
What’s Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders.
In ACM International Conference on Knowledge Discovery and Data Mining (KDD), pages 1268–1279, 2023.
- [19]
T. Li, H. Chang, S. Mishra, H. Zhang, D. Katabi, and D. Krishnan.
MAGE: Masked generative encoder to unify representation learning and image synthesis.
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
- [20]
T. Li, Y. Tian, H. Li, M. Deng, and K. He.
Autoregressive Image Generation without Vector Quantization.
arXiv preprint arXiv:2406.11838, 2024.
- [21]
X. Li, X. Li, L. Chen, X. Zhang, M. Yuan, and J. Wang.
Circuit Transformer: End-to-end Circuit Design by Predicting the Next Gate.
arXiv preprint arXiv:2403.13838, 2024.
- [22]
H. Liu, K. Simonyan, and Y. Yang.
DARTS: Differentiable Architecture Search.
arXiv preprint arXiv:1806.09055, 2018.
- [23]
A. Mishchenko, S. Chatterjee, L. Amarú, E. Testa, and W. Lau Neto.
IWLS 2022 Programming Contest.
[https://github.com/alanminko/iwls2022-ls-contest/](https://github.com/alanminko/iwls2022-ls-contest/), 2022.
- [24]
OpenAI.
GPT-4 Technical Report.
arXiv preprint arXiv:2303.08774, 2023.
- [25]
R. Rudell.
Espresso-mv: Algorithms for multiple-valued logic minimization.
In Proceedings of IEEE Custom Integrated Circuit Conference, pages 230–234, 1985.
- [26]
M. Thomas and A. T. Joy.
Elements of information theory.
Wiley-Interscience, 2006.
- [27]
K. Tian, Y. Jiang, Z. Yuan, B. Peng, and L. Wang.
Visual autoregressive modeling: Scalable image generation via next-scale prediction.
In Annual Conference on Neural Information Processing Systems (NIPS), 2024.
- [28]
D. Tsaras, A. Grosnit, L. Chen, Z. Xie, H. Bou-Ammar, and M. Yuan.
ShortCircuit: AlphaZero-Driven Circuit Design.
arXiv preprint arXiv:2408.09858, 2024.
- [29]
Y. Umuroglu, Y. Akhauri, N. J. Fraser, and M. Blott.
Logicnets: Co-designed neural networks and circuits for extreme-throughput applications.
In International Conference on Field-Programmable Logic and Applications (FPL), pages 291–297. IEEE, 2020.
- [30]
A. Van Den Oord, O. Vinyals, et al.
Neural Discrete Representation Learning.
In Annual Conference on Neural Information Processing Systems (NIPS), 2017.
- [31]
Z. Wang, J. Wang, Q. Yang, Y. Bai, X. Li, L. Chen, H. Jianye, M. Yuan, B. Li, Y. Zhang, et al.
Towards Next-Generation Logic Synthesis: A Scalable Neural Circuit Generation Framework.
In Annual Conference on Neural Information Processing Systems (NIPS), 2024.
- [32]
C. Wolf, J. Glaser, and J. Kepler.
Yosys-a free verilog synthesis suite.
In Proceedings of the 21st Austrian Workshop on Microelectronics (Austrochip), volume 97, 2013.
- [33]
H. Wu, H. Zheng, Y. Pu, and B. Yu.
Circuit Representation Learning with Masked Gate Modeling and Verilog-AIG Alignment.
In International Conference on Learning Representations (ICLR), 2025.
- [34]
C. Ying, T. Cai, S. Luo, S. Zheng, G. Ke, D. He, Y. Shen, and T.-Y. Liu.
Do transformers really perform badly for graph representation?
In Annual Conference on Neural Information Processing Systems (NIPS), 2021.
- [35]
J. Yu, X. Li, J. Y. Koh, H. Zhang, R. Pang, J. Qin, A. Ku, Y. Xu, J. Baldridge, and Y. Wu.
Vector-Quantized Image Modeling with Improved VQGAN.
arXiv preprint arXiv:2110.04627, 2021.
- [36]
X. Zhou, X. Li, Y. Lian, Y. Wang, L. Chen, M. Yuan, J. Hao, G. Chen, and P. A. Heng.
SeaDAG: Semi-autoregressive Diffusion for Conditional Directed Acyclic Graph Generation.
arXiv preprint arXiv:2410.16119, 2024.

## Appendix A Related Works

### A.1 Autoregressive Modeling

The autoregressive modeling paradigm [24, 27] has been widely adopted for generation tasks in language and vision domains.
Built on the transformer architecture, autoregressive models are commonly implemented using causal attention mechanisms in language domains [24], which process data sequentially.
However, information does not inherently require sequential processing in vision and graph generation tasks.
To address this, researchers have employed bidirectional attention mechanisms for autoregressive modeling [20, 27, 7, 19].
This approach predicts the next token based on previously predicted tokens while allowing unrestricted communication between tokens, thereby relaxing the sequential constraints of traditional autoregressive methods.
In this paper, we adopt masked autoregressive modeling for circuit generation, leveraging its ability to provide a global perspective and enhance the modeling of complex dependencies.

### A.2 Circuit Generation

In addition to the DAS-based approaches, researchers have also explored next-gate prediction techniques inspired by LLMs for circuit generation.
Circuit Transformer [21] predicts the next logic gate using a depth-first traversal and equivalence-preserving decoding.
SeaDAG [36] employs a semi-autoregressive diffusion model for DAG generation, maintaining graph structure for precise control.
ShortCircuit [28] uses a transformer to generate Boolean expressions from truth tables via next-token prediction.
However, these methods are limited by their global view, restricting circuit size and failing to reduce the search space.
In contrast, our approach uses global-view masked autoregressive decoding to generate circuits while ensuring logical equivalence and significantly reducing the search space during the DAS process.

## Appendix B Implementation Details

### B.1 CircuitVQ

The training process of CircuitVQ employs a linear learning rate schedule with the AdamW optimizer set at a learning rate of $2\times 10^{-4}$, a weight decay of 0.1, and a batch size of 2048.
The model is fine-tuned for 20 epochs on 8$\times$A100 GPUs with 80G memory each.
Moreover, we use the Graphormer [34] as our CircuitVQ architecture, as mentioned before.
Specifically, CircuitVQ comprises 6 encoder layers and 1 decoder layer.
The hidden dimension and FFN intermediate dimension are 1152 and 3072, respectively.
Additionally, the multi-head attention mechanism employs 32 attention heads.
For the vector quantizer component, the codebook dimensionality is set to 4 to improve the codebook utilization, and the codebook size is configured to 8192.

### B.2 CircuitAR

The training process of CircuitAR employs a linear learning rate schedule with the AdamW optimizer set at a learning rate of $2\times 10^{-4}$, a weight decay of 0.1, and a batch size of 4096.
The model is fine-tuned for 20 epochs on 16$\times$A100 GPUs with 80G memory each.
Moreover, we use the Transformer variant of Llama [9] as our CircuitAR architecture as mentioned before.
To form different model sizes, we vary the hidden dimension, FFN intermediate dimension, number of heads and number of layers.
We present the details of our CircuitAR architecture configurations in [Table 3](https://arxiv.org/html/2502.12751v2#A2.T3).
For the rest of the hyperparameters, we keep them the same as the standard Llama model.

**Table 3: Model architecture configurations of CircuitAR.**
|  | hidden dim | FFN dim | heads | layers |
| --- | --- | --- | --- | --- |
| CircuitAR-300M | 1280 | 3072 | 16 | 16 |
| CircuitAR-600M | 1536 | 4096 | 16 | 20 |
| CircuitAR-1B | 1800 | 6000 | 24 | 24 |
| CircuitAR-2B | 2048 | 8448 | 32 | 30 |

## Appendix C Training Datasets

This section presents a multi-output subcircuit extraction algorithm designed for generating training datasets.
The algorithm processes circuits represented in the And Inverter Graph (AIG) format by iterating over each non-PI node as a pivot node.
The extraction process consists of three key stages:

- 1.
Single-Output Subcircuit Extraction.
The algorithm extracts single-output subcircuits by analyzing the transitive fan-in of the pivot node.
The transitive fan-in includes all nodes that influence the output of the pivot node, encompassing both direct predecessors and nodes that propagate signals to it.
The extraction process employs a breath-first search (BFS) algorithm, constrained by a maximum input size, to ensure comprehensive coverage of relevant nodes associated with the pivot node.
- 2.
Multi-Output Subcircuit Expansion.
Single-output subcircuits are expanded into multi-output subcircuits through transitive fan-out exploration.
The transitive fan-out comprises all nodes influenced by the pivot node, including immediate successors and downstream nodes reachable through signal propagation.
This expansion captures the broader network of nodes that either influence or are influenced by the subcircuits of the pivot node.
- 3.
Truth Table Generation.
The algorithm computes truth tables for the extracted subcircuits to serve as training labels.
Additionally, these truth tables help identify functionally equivalent subcircuits.
Recognizing these equivalences is essential, as it can lead to data imbalance in the training set.

To mitigate data imbalance, a constraint is imposed, limiting each truth table to at most $M$ distinct graph topologies.
For truth tables with fewer than $M$ representations, logic synthesis techniques (specifically rewriting algorithms) are applied to generate functionally equivalent subcircuits with distinct topologies.
This approach ensures topological diversity while maintaining functional equivalence.
Finally, the training datasets with around 400000 circuits (average 200 gates per circuit) are generated using circuits from the ISCAS’85 [6], IWLS’05 [2], and EPFL [3].
$M$ is set to 5 during the generation process.
The sizes of PI and PO are capped at 15 each in the training dataset, ensuring manageable truth table sizes while maintaining complexity.

## Appendix D Data Augmentation

Following dataset generation, we identified that the data volume was still insufficient.
To address this limitation, we implemented data augmentation techniques.
Leveraging the topological invariance of graphs, we randomly shuffled the order of graph nodes, as this operation does not alter the underlying structure of the circuit.
Furthermore, since inserting idle nodes preserves the circuit structure, we randomly introduced idle nodes into the graphs.
The proportion of idle nodes is randomly selected ranging from 0% to 80%.
Moreover, incorporating idle nodes enables CircuitAR to identify which nodes can remain inactive for a fixed number of nodes.
This allows CircuitAR to generate circuits logically equivalent to the truth table while utilizing fewer graph nodes.
This strategy can improve CircuitAR’s efficiency and enhance its generalizability.

Figure: Figure 9: The impact of idle NAND gates on BitsD for CircuitAR with different ratios of isolated NAND gates.
Refer to caption: x8.png

## Appendix E Idle NAND Gates

As shown in [Figure 9](https://arxiv.org/html/2502.12751v2#A4.F9), all models exhibit a gradual decline in BitsD as the isolated gates proportion increases from 0% to 45%.
Large model with 2B parameters demonstrates significantly greater robustness, maintaining BitsD values within a narrow range across varying isolation ratios.
In contrast, the small model with 300M parameters shows a more pronounced degradation, with BitsD increasing from 0.5317 to 0.5484 under the same conditions.
This disparity highlights the enhanced ability of larger models to efficiently utilize NAND gates for implementing the same truth table.
The consistently low BitsD observed in the 2B model underscores its practical utility in predefining search spaces for DAS, offering a notable advantage over smaller models.

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 6
presentation: 4
questions: * How does the overall logic optimization system with this approach compare to classic logic optimization algorithms such as resyn2, for various circuit sizes?
soundness: 3
strengths: * Novel modeling approach of circuit generation from truth table using a masked autoregressive graph model.
summary: This work presents an approach for circuit generation that combines conditional generative modeling with differentiable architecture search (DAS). The CircuitAR model produces initial circuits from truth tables which is optimized by DAS. The CircuitAR graph autoregressive model is trained with masked autoregressive training conditioned on truth tables that define the circuit functionality.
weaknesses: * The primary utility of this work is reducing DAS search steps. This is a weaker utility compared to improving the quality of circuit optimization (fewer nodes or lower area/delay/power).
* Quality comparison versus classical logic optimization algorithms such as resyn2 is not presented. Prior work such as ShortCircuit presents such comparisons.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: - In line 161, $\ell_2(\cdot)$ seems to represent a $\ell_2$-normalized vector, rather than the $\ell_2$-norm of a vector. Maybe this can be mentioned in an explicit way to avoid confusion.
soundness: 3
strengths: - The "truth table - discrete tokens - adjacency matrix" pipeline is novel and technically solid.
- The result shows that the performance of CircuitAR can scale with more parameters and tokens, which looks promising.
summary: This paper presents an autoregressive approach to generate (raw) circuits from truth tables, which can serve as an initialization of differentiable architecture search to generate valid and exactly equivalent circuits. Firstly, an autoencoder model "CircuitVQ"  is trained to achieve interconversion between a circuit and discrete tokens. Then, an autoregressive model "CircuitAR" is trained to generate tokens from truth table. During the inference stage, given a truth table, the CircuitAR model transform it into tokens, and the decoder of CircuitVQ further transform the tokens into a reconstructed circuit (represented by an adjacency matrix). The matrix is further processed in a greedy way to make it valid (no cycles, etc), and serve as an initialization of CircuitNN for an exactly equivalent circuit. Experimental result shows that the performance of CircuitAR can scale with more parameters and tokens, and the proposed pipeline performs better than baseline approaches.
weaknesses: - The circuit size scalability of the proposed approach is questionable. No experimental result is shown about the circuit size scalability. The paper mentioned "a tenfold increase in circuit complexity compared to prior works", but it is not clear how this conclusion is drawn, and how the "circuit complexity" is measured (number of gates/PIs/POs?). It looks like all the circuits selected in table 1 and 2 are pretty small.
- The number of sample circuits for evaluation is very small. In table 1 and 2, only ten circuits are included. Even if there is a scalability issue, the test circuits can be extended by extracting sub-circuits from large benchmark circuits.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1.	BitsD validation: Can you show correlation plots between BitsD and (a) DAS steps, (b) final #NAND / mapped area, (c) success under fixed time budgets across a large set? Does BitsD remain predictive when node budgets and idle gate ratios vary? 

2.	Truth table representation: How is T encoded for cross attention in CircuitAR?   

3.	Broader benchmarks: Can you include ABC/Yosys and report mapped area/timing under a standard cell library? Even if not directly comparable, this gives a reality check on hardware relevance.   

4.	Statistical rigor: Please provide error bars (multiple seeds) for Tables 1–2 and Figures 4–7; clarify stopping criteria and early stopping policies for DAS.   

5.	Search space definition: How exactly is the important metric, “Search Space” in Table 2, computed? A precise definition would help the community reproduce and interpret the gains.   

6.	Artifacts: Will you release code, data gen scripts, and pretrained models?
soundness: 3
strengths: • A clear and interesting integration of masked autoregressive generation with DAS. The idea of using a generative prior to prune/guide the continuous search is sensible and novel in this space. 

• The training/evaluation pipeline is well illustrated, and the DAG projection routine is explicitly provided. 

• Empirical signal: Across ten IWLS’22 circuits, the approach reports substantial reductions in DAS steps and non trivial #NAND savings while keeping functional correctness. Scaling experiments and an ablation on using CircuitAR derived edge probabilities provide some evidence that larger models and the proposed initialization matter.
summary: This paper proposes a two-stage approach to logic circuit generation from truth tables. The authors introduce Bits Distance—the MAE between an untrained DAS model (initialized by CircuitAR) and the target truth table—as a proxy for conditional generation quality.
Empirically, on selected IWLS’22 tasks, the method claims 100% functional accuracy while reducing DAS optimization steps by ~45% on average vs. CircuitNN and search space / #NAND relative to baselines. The paper also presents scaling curves (300M→2B parameters) for BitsD and an “emergent capability” plot suggesting deeper yet more NAND efficient circuits at 2B parameters. The Limitations section acknowledges compute constraints.
weaknesses: •	The main claims rely on 10 IWLS’22 circuits (Table 1–2, p.8). This is a small sample of the broader IWLS/EPFL/ISCAS landscape. 

•	There is no comparison to industrial strength flows (e.g., ABC/Yosys scripts such as resyn2) in terms of final area/depth/delay under a standard cell library—even as a reference. The paper frames #NAND and “search space” but not mapped area/timing—which is what designers ultimately care about.  

•	BitsD is measured as MAE of an untrained CircuitNN initialized by CircuitAR vs. truth table outputs. While intuitively appealing as a pre DAS proxy, the paper does not provide correlation analyses between BitsD and the final outcomes designers care about (e.g., DAS steps, final #NAND/area, or success under time limits). 

•	CircuitAR conditions on full truth tables via cross attention, but the paper does not describe how T is represented (row major? compressed? learned embeddings?). The training data caps PI/PO at ≤15, which is small compared to real designs; beyond this, truth table representations become exponential in PI. The method’s practical deployment arguably hinges on decomposition, but the paper does not implement or evaluate a decomposition→recombination pipeline. 

•	The “phase transition” at 2B parameters is based on four model sizes and two metrics (depth/#NAND), without uncertainty bands or tests for statistical significance. Also, Figures 4–6 (p.7) assert power law like improvements in BitsD, but again no error bars or replicates are shown. 

•	While Appendix B gives hyperparameters and hardware, the paper does not clearly state that code, models, and the circuit dataset will be released. Given the complexity and compute, artifact availability is essential for verification.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1. (**Major**) Could you provide results on larger or full IWLS circuits, including functional accuracy and final area/NAND count after DAS? Please report success rate and scaling behavior with circuit size.

2. What is the empirical correlation between Bits Distance and post-DAS metrics (NAND count, search steps, area)?

3. How consistent is the claim of “100% correctness guaranteed by DAS”? Please provide convergence thresholds, number of restarts, and any failure cases.

4. Could you include training/inference cost statistics (GPU hours, DAS runtime per circuit) to assess practical efficiency?
soundness: 3
strengths: 1. Clear motivation and practical pipeline combining generation and optimization.

2. Solid empirical study showing efficiency and scaling trends.

3. Comprehensive ablations and architectural details supporting reproducibility.
summary: The paper proposes Architect of the Bits World, a framework that combines a conditional generative model with differentiable architecture search (DAS) for logic-circuit generation. A VQ-based tokenizer (CircuitVQ) converts circuits into discrete tokens, and a masked autoregressive Transformer (CircuitAR) generates circuit topologies conditioned on truth tables. The generated circuits serve as initialization priors for DAS, which guarantees final functional correctness. The authors introduce a proxy metric (Bits Distance) to measure conditional generation quality and demonstrate that larger models (0.3B–2B params) reduce DAS search steps and NAND counts on IWLS benchmarks, showing promising scaling and efficiency.
weaknesses: 1. Limited evaluation on large IWLS circuits; scalability and generalization remain unproven.

2. BitsD is only heuristically justified as a proxy for final metrics.

3. Lack of analysis on DAS convergence and potential failure cases.


Decision: Reject