Quantum Architecture Search With Unsupervised Representation Learning

## Abstract

Abstract Unsupervised representation learning presents new opportunities for advancing Quantum Architecture Search (QAS) on Noisy Intermediate-Scale Quantum (NISQ) devices. QAS is designed to optimize quantum circuits for Variational Quantum Algorithms (VQAs). Most QAS algorithms tightly couple the search space and search algorithm, typically requiring the evaluation of numerous quantum circuits, resulting in high computational costs and limiting scalability to larger quantum circuits. Predictor-based QAS algorithms mitigate this issue by estimating circuit performance based on structure or embedding. However, these methods often demand time-intensive labeling to optimize gate parameters across many circuits, which is crucial for training accurate predictors. Inspired by the classical neural architecture search algorithm Arch2vec , we investigate the potential of unsupervised representation learning for QAS without relying on predictors. Our framework decouples unsupervised architecture representation learning from the search process, enabling the learned representations to be applied across various downstream tasks. Additionally, it integrates an improved quantum circuit graph encoding scheme, addressing the limitations of existing representations and enhancing search efficiency. This predictor-free approach removes the need for large labeled datasets. During the search, we employ REINFORCE and Bayesian Optimization to explore the latent representation space and compare their performance against baseline methods. Our results demonstrate that the framework efficiently identifies high-performing quantum circuits with fewer search iterations.

## 1 Introduction

Quantum Computing (QC) has made significant progress over the past decades. Advances in quantum hardware and new quantum algorithms have demonstrated potential advantages (Stein et al., ) over classical computers in various tasks, such as image processing (Wang et al., ), reinforcement learning (Skolik et al., ), knowledge graph embedding (Ma et al., ), and network architecture search (Zhang et al., ; Giovagnoli et al., ; Du et al., ). However, the scale of quantum computers is still limited by environmental noise, which leads to unstable performance. These noisy intermediate-scale quantum (NISQ) devices lack fault tolerance, which is not expected to be achieved in the near future (Preskill, ). The variational quantum algorithm (VQA), a hybrid quantum algorithm that utilizes quantum operations with adjustable parameters, is considered a leading strategy in the NISQ era (Cerezo et al., ). In VQA, the parameterized quantum circuit (PQC) with trainable parameters is viewed as a general paradigm of quantum neural networks and has achieved notable success in quantum machine learning. These parameters control quantum circuit operations, adjusting the distribution of circuit output states, and are updated by a classical optimizer based on a task-specific objective function. Although VQA faces challenges such as Barren Plateaus (BP) and scalability issues, it has demonstrated the potential to improve performance across various domains, including image processing, combinatorial optimization, chemistry, and physics (Pramanik et al., ; Amaro et al., ; Tilly et al., ). One example of a VQA is the variational quantum eigensolver (VQE) (Peruzzo et al., ; Tilly et al., ), which approximates the ground state and offers flexibility for quantum machine learning. We are considering using VQE to evaluate the performance of certain quantum circuits.

Unsupervised representation learning seeks to discover hidden patterns or structures within unlabeled data, a well-studied problem in computer vision research (Radford et al., ). One common approach is the autoencoder, which is effective for feature representation. It consists of an encoder and decoder, which first maps images into a compact feature space and then decodes them to reconstruct similar images. Beyond images, autoencoders can also learn useful features from graphs, such as encoding and reconstructing directed acyclic graphs (DAGs) or neural network architectures (Yan et al., ; Zhang et al., ; Pan et al., ; Wang et al., ). In most research, architecture search and representation learning are coupled, which results in inefficient searches heavily dependent on labeled architectures that require numerous evaluations. The Arch2vec framework aims to decouple representation learning from architecture search, allowing downstream search algorithms to operate independently (Yan et al., ). This decoupling leads to a smooth latent space that benefits various search algorithms without requiring extensive labeling.

Quantum architecture search (QAS) or quantum circuit architecture search is a framework for designing quantum circuits efficiently and automatically, aiming to optimize circuit performance (Du et al., ). Various algorithms have been proposed for QAS (Zhang et al., ; Du et al., ; Zhang et al., ; He et al., ; Giovagnoli et al., ). However, most algorithms combine the search space and search algorithm, leading to inefficiency and high evaluation costs. The effectiveness of the search algorithm often depends on how well the search space is defined, embedded, and learned. Finding a suitable circuit typically requires evaluating different architectures many times. Although predictor-based QAS He et al. () can separate representation learning from the search algorithm, it often relies on labeling different architectures via evaluation, and the training performance depends heavily on the quantity and quality of evaluations and the embedding. In this work, we are inspired by the idea of decoupling, and we aim to conduct QAS without labeling. We seek to explore whether decoupling can embed quantum circuit architectures into a smooth latent space, benefiting predictor-free QAS algorithms.We summarise our contributions as follows:

- •
We have successfully incorporated decoupling into unsupervised architecture representation learning within QAS, significantly improving search efficiency and scalability. By applying REINFORCE and Bayesian optimization directly to the latent representation, we eliminate the need for a predictor trained on large labeled datasets, thereby reducing prediction uncertainty.
- •
Our proposed quantum circuit encoding scheme overcomes limitations in existing representations, enhancing search performance by providing more accurate and effective embeddings.
- •
Extensive experiments on quantum machine learning tasks, including quantum state preparation, max-cut, and quantum chemistry (Liang et al., ; Poljak & Rendl, ; Tilly et al., ), confirm the effectiveness of our framework. The pre-trained quantum architecture embeddings significantly enhance QAS across these applications.

## 2 Related Work

##### Unsupervised Graph Representation Learning.

Graph data is becoming a crucial tool for understanding complex interactions between real-world entities, such as biochemical molecules (Jiang et al., ), social networks (Shen et al., ), purchase networks from e-commerce platforms (Li et al., ), and academic collaboration networks (Newman, ). Graphs are typically represented as discrete data structures, making it challenging to solve downstream tasks due to large search spaces.
Our work focuses on unsupervised graph representation learning, which seeks to embed graphs into low-dimensional, compact, and continuous representations without supervision while preserving the topological structure and node attributes. In this domain, approaches such as those proposed by Perozzi et al. (); Wang et al. (); Grover & Leskovec (); Tang et al. () use local random walk statistics or matrix factorization-based objectives to learn graph representations. Alternatively, methods like Kipf & Welling (); Hamilton et al. () reconstruct the graph’s adjacency matrix by predicting edge existence, while others, such as Veličković et al. (); Sun et al. (); Peng et al. (), maximize the mutual information between local node representations and pooled graph representations.
Additionally, Xu et al. () investigate the expressiveness of Graph Neural Networks (GNNs) in distinguishing between different graphs and introduce Graph Isomorphism Networks (GINs), which are shown to be as powerful as the Weisfeiler-Lehman test (Leman & Weisfeiler, ) for graph isomorphism. Inspired by the success of Arch2vec (Yan et al., ), which employs unsupervised graph representation learning for classical neural architecture search (NAS), we adopt GINs to injectively encode quantum architecture structures, as quantum circuit architectures can also be represented as DAGs.

##### Quantum Architecture Search (QAS).

As discussed in the previous section, PQCs are essential as ansatz for various VQAs (Benedetti et al., ). The expressive power and entangling capacity of PQCs play a crucial role in their optimization performance (Sim et al., ). Poorly designed ansatz can suffer from limited expressive power or entangling capacity, making it difficult to reach the global minimum for an optimization problem. Moreover, such ansatz may be more prone to noise (Stilck França & Garcia-Patron, ), inefficiently utilize quantum resources, or lead to barren plateaus that hinder the optimization process (McClean et al., ; Wang et al., ). To address these challenges, QAS has been proposed as a systematic approach to identify optimal PQCs. The goal of QAS is to efficiently and effectively search for high-performance quantum circuits tailored to specific problems, minimizing the loss functions while adhering to constraints such as hardware qubit connections, native quantum gate sets, quantum noise models, training loss landscapes, and other practical considerations. Quantum architectures share many properties with neural network architectures, such as hierarchical, directed, and acyclic structures. As a result, QAS methods have been heavily inspired by techniques from NAS. Specifically, approaches such as greedy algorithms (Mitarai et al., ; Tang et al., ), evolutionary or genetic methods (Zhang & Zhao, ; Ding & Spector, ), RL-based engines (Kuo et al., ; Ostaszewski et al., ), Bayesian optimization (Duong et al., ), and gradient-based methods (Zhang et al., ) have all been employed to discover improved PQCs for VQAs. However, these methods require the evaluation of numerous quantum circuits, which is both time-consuming and computationally expensive. To mitigate this issue, predictor-based approaches (Zhang et al., ; He et al., ) have been introduced, but they also face limitations. These approaches rely on large sets of labeled circuits to train predictors with generalized capabilities and introduce additional uncertainty into the search process, necessitating the reevaluation of candidate circuits. In this work, we propose a framework aimed at further addressing these challenges.

## 3 QAS with Unsupervised Representation Learning

Figure: (a) Architecture encoding scheme
Refer to caption: extracted/5949113/images/encoding3.drawio.png

In this work, we present our method, as illustrated in Figure 1, which consists of two independent learning components: an autoencoder for circuit architecture representation learning, and a search process that includes both search and evaluation strategies. The search space is defined by the number of gates in a circuit and an operation pool comprising general gate types such as X, Y, Z, H, Rx, Ry, Rz, U3, CNOT, CY, CZ. A random generator creates a set of circuit architectures based on predefined parameters, including the number of qubits, the number of gates, and the maximum circuit depth. These architectures are then encoded into two matrices and input into the autoencoder. The autoencoder independently learns a latent distribution from the search space and produces pre-trained architecture embeddings for the search algorithms. The evaluation strategy takes the circuit architectures generated by the search algorithm and returns a performance assessment. For evaluating circuit architectures, we use the ground state of a Hamiltonian for max-cut and quantum chemistry problems, and fidelity for quantum state preparation tasks.

### 3.1 Circuit Encoding Scheme

We represent quantum circuits as DAGs using the circuit encoding scheme $\mathcal{E}^{GSQAS}$, as described in He et al. (; ). Each circuit is transformed into a DAG by mapping the gates on each qubit to a sequence of nodes, with two additional nodes added to indicate the input and output of circuits. The resulting DAG is described by an adjacency matrix, as shown in Figure 1(a). The set of nodes is further characterized by a gate matrix, which shows the node features including position information.

However, the encoding scheme $\mathcal{E}^{GSQAS}$ represents all occupied qubits as $1$ without distinguishing between the control and target positions of two-qubit gates, which limits the effectiveness of circuit representation learning and leads to confusion during circuit reconstruction. Additionally, the adjacency matrix weights do not accurately reflect the original gate connections. To address these limitations, we propose a new encoding scheme. In our method, we explicitly encode positional information for two-qubit gates, such as CNOT and CZ, by assigning $-1$ to the control qubit and $1$ to the target qubit. Furthermore, we represent the number of qubits involved in an edge as the connection weights in the adjacency matrix, as shown in Figure 1(a). These modifications enhance circuit representation learning and improve the overall effectiveness of the search.

### 3.2 Variational Graph Isomorphism Autoencoder

#### 3.2.1 Preliminaries

The most common graph autoencoders (GAEs) consist of an encoder and a decoder, where the encoder maps a graph into a feature space, and the decoder reconstructs the graph from those features. One prominent example is the variational graph autoencoder (VGAE), a promising framework for unsupervised graph representation learning that utilizes a graph convolutional network as its encoder and a simple inner product as its decoder (Kipf & Welling, ). In this work, however, we do not employ the common VGAE as a framework for learning latent representations. Instead, we utilize a more powerful encoder GIN (Xu et al., ).

###### Definition 1 .

We are given a circuit created by $m$ gate types, $h$ gates and $g$ qubits. Then, the circuit can be described by a DAG $G=\{V,E\}$ with $n=h+2=|V|$ gate nodes including START and END. The adjacency matrix of graph $G$ is summarized in $n\times n$ matrix $A$ and its gate matrix $X$ is in size of $n\times(m+2+g)$. We further introduce $d$-dimensional latent variables $z_{i}$ composing latent matrix $Z=\{z_{1},..,z_{K}\}^{T}$.

###### Definition 1 .

#### 3.2.2 Encoder

The encoder GIN maps the structure and node features to latent representations $Z$. An approximation of the posterior distribution $q(Z|X,A)$ is:

$$ $\displaystyle q(Z|X,A)=\prod^{K}_{i=1}q(z_{i}|X,A)\text{,}$ (1) $$

where $q(z_{i}|X,A)=\mathcal{N}(z_{i}|\mu_{i},\text{diag}(\sigma^{2}_{i}))$.
The $L$-layer GIN generates the embedding matrix $M^{(s)}$ for $s$-layer by:

$$ $\displaystyle M^{(s)}=MLP^{(s)}((1+\epsilon^{(s)})\cdot M^{(s-1)}+\hat{A}M^{(s -1)}),s=1,2,...,L\text{,}$ (2) $$

Where $M^{(0)}=X$, and $\epsilon^{(s)}$ is a bias with a standard normal distribution for each layer. The $MLP$ is a multi-layer perceptron consisting of Linear-BatchNorm-LeakyReLU layers, and $\hat{A}=A+A^{T}$ transforms a directed graph into an undirected one to capture bi-directional information. In this work, we introduce a new fusion layer, a fully connected layer that aggregates feature information from all GIN layers, rather than just the last one. The mean $\mu=\text{GIN}_{\mu}(X,\hat{A})=FC_{1}(M^{(L)})$ is computed using the fully connected layer $FC_{1}$, and similarly, the standard deviation $\sigma$ is computed via $FC_{2}$. We can then sample the latent matrix $Z\sim q(Z|X,A)$ by $z_{i}=\mu_{i}+\sigma_{i}\cdot\epsilon_{i}$.
For all experiments, we use $L=5$ GIN layers, a 16-dimensional latent vector $z_{i}$, and a GIN encoder with hidden sizes of 128. More details on the hyperparameters can be found in Appendix A.3.

#### 3.2.3 Decoder

The decoder takes the sampled latent variables $Z$ as input to reconstruct both the adjacency matrix $A$ and the gate matrix $X=[X^{t},X^{q}]$, where $X^{t}$ encodes the gate types and $X^{q}$ encodes the qubits on which the gates act. The generative process is summarized as follows:

$$ $\displaystyle p(A|Z)$ $\displaystyle=\prod^{K}_{i=1}\prod^{K}_{j=1}p(A_{ij}|z_{i},z_{j}),\text{ with }p(A_{ij}|z_{i},z_{j})=\text{ReLU}_{j}(F_{1}(z^{T}_{i}z_{j}))\text{,}$ (3) $\displaystyle p(X|Z)$ $\displaystyle=\prod^{K}_{i=1}p(x_{i}|z_{i})\text{,}\text{ with }p(x^{t}_{i}|z_ {i})=\text{softmax}(F_{2}(z_{i}))\text{, }p(x^{q}_{i}|z_{i})=\text{tanh}(F_{2} (z_{i}))\text{,}$ (4) $$

where both $F_{1}$ and $F_{2}$ are trainable linear functions.

#### 3.2.4 Objective Function

The weights in the encoder and decoder are optimized by maximizing the evidence lower bound (ELBO) $\mathcal{L}$, which is defined as:

$$ $\displaystyle\mathcal{L}=E_{q(Z|X,A)}[\log p(X^{\text{type}},X^{\text{qubit}}, A|Z)]-\text{KL}[(q(Z|X,A))||p(Z)]\text{,}$ (5) $$

where KL$[q(\cdot)||p(\cdot)]$ represents the Kullback-Leibler (KL) divergence between $q(\cdot)$ and $p(\cdot)$. We further adopt a Gaussian prior $p(Z)=\prod_{i}\mathcal{N}(z_{i}|0,I)$. The weights are optimized using minibatch gradient descent, with a batch size of 32.

### 3.3 Architecture Search Strategies

#### 3.3.1 Reinforcement Learning (RL)

After conducting initial trials with PPO (Schulman et al., ) and A2C (Huang et al., ), we adopt REINFORCE (Williams, ) as a more effective reinforcement learning algorithm for architecture search. In this approach, the environment’s state space consists of pre-trained embeddings, and the agent uses a one-cell LSTM as its policy network. The agent selects an action, corresponding to a sampled latent vector based on the distribution of the current state, and transitions to the next state based on the chosen action. The reward for max-cut and quantum chemistry tasks is defined as the ratio of energy to ground energy, with values outside the range [0, 1] clipped to 0 or 1. For the state preparation task, circuit fidelity is used as the reward.
We employ an adaptive batch size, with the number of steps per training epoch determined by the average reward of the previous epoch. Additionally, we use a linear adaptive baseline, defined by the formula $B=\alpha\cdot B+(1-\alpha)\cdot R_{avg}$, where $B$ denotes the baseline, $\alpha$ is a predefined value in the range [0,1], and $R_{avg}$ is the average reward. Each run in this work involves 1000 searches.

#### 3.3.2 Bayesian Optimization (BO)

As another search strategy used in this work without labeling, we employ Deep Networks for Global Optimization (DNGO)(Snoek et al., ) in the context of BO. We adopt a one-layer adaptive BO regression model with a basis function extracted from a feed-forward neural network, consisting of 128 units in the hidden layer, to model distributions over functions. Expected Improvement (EI)(Mockus, ) is selected as the acquisition function. EI identifies the top-k embeddings for each training epoch, with a default objective value of 0.9. The training begins with an initial set of 16 samples, and in each subsequent epoch, the top-k architectures proposed by EI are added to the batch. The network is retrained for 100 epochs using the architectures from the updated batch. This process is iterated until the predefined number of search iterations is reached.

## 4 Experimental Results

To demonstrate the effectiveness and generalization capability of our approach, we conduct experiments on three well-known quantum computing applications: quantum state preparation, max-cut, and quantum chemistry. For each application, we start with a simple example involving 4 qubits and then progress to a more complex example with 8 qubits. We utilize a random generator to create 100,000 circuits as the search space, and all experiments are performed on a noise-free simulator during the search process. Detailed settings are provided in Appendix A.2. We begin by evaluating the model’s pre-training performance for unsupervised representation learning (§4.1), followed by an assessment of QAS performance based on the pre-trained latent representations (§4.2).

### 4.1 Pre-training Performance

Observation (1): GAE and VGAE (Kipf & Welling, ) are two popular baselines for NAS. In an attempt to find models capable of capturing superior latent representations of quantum circuit architectures, we initially applied these two well-known models. However, due to the increased complexity of quantum circuit architectures compared to neural network architectures, these models failed to deliver the expected results. In contrast, models based on GINs (Xu et al., ) successfully obtained valid latent representations, attributed to their more effective neighbor aggregation scheme.
Table 1 presents a performance comparison between the original model using the $\mathcal{E}^{GSQAS}$ encoding and the improved model with our enhanced encoding for 4, 8, and 12 qubit circuits, evaluated across five metrics: Accuracy${ops}$, which measures the reconstruction accuracy of gate types in the gate matrix for the held-out test set; Accuracy${qubit}$, which reflects the reconstruction accuracy of qubits that the gates act on; Accuracy${adj}$, which measures the reconstruction accuracy of the adjacency matrix; Falpos${mean}$, which represents the mean false positives in the reconstructed adjacency matrix; and KLD (KL divergence), which indicates the continuity and smoothness of the latent representation.
The results in the table indicate that the improved model with our enhanced encoding achieves comparable or better than the original. This improvement can be attributed to two factors: first, the new encoding better captures the specific characteristics of the circuits, and second, the fusion of outputs from multiple layers of GIN helps retain shallow information, resulting in more stable training.

**Table 1: Pretraining model performance of 4-, 8-, and 12-qubit circuits across the four metrics.**
| Qubit | Model | Metric |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Accuracyops | Accuracyqubit | Accuracyadj | Falposmean | KLD |  |  |
| 4 | GSQAS | 99.99 | 99.99 | 99.91 | 100.00 | 0.061 |
| 4 | Ours | 100 | 99.97 | 98.89 | 23.41 | 0.045 |
| 8 | GSQAS | 86.69 | 99.98 | 99.82 | 100.00 | 0.038 |
| 8 | Ours | 100 | 98.65 | 97.34 | 7.35 | 0.029 |
| 12 | GSQAS | 86.69 | 99.94 | 99.70 | 100.00 | 0.028 |
| 12 | Ours | 98.67 | 99.14 | 97.79 | 4.75 | 0.022 |

Observation (2): In Figure 2, we employ two popular techniques, PCA (Shlens, ) and t-SNE (Van der Maaten & Hinton, ), to visualize the high-dimensional latent representations of 4- and 12-qubit quantum machine learning (QML) applications based on our pre-trained models. The results highlight the effectiveness of our new encoding approach for unsupervised clustering and high-dimensional data visualization. The figures show that the latent representation space of quantum circuits is smooth and compact, with architectures of similar performance clustering together when the search space is limited to 4 qubits. Notably, high-performance quantum circuit architectures are concentrated on the right side of the visualizations. In particular, PCA yields exceptionally smooth and compact representations with strong clustering effects, making it easier and more efficient to conduct QAS within such a structured latent space. This provides a robust foundation for our QAS algorithms.

For the 12-qubit latent space, high-performance circuits (shown in red) are less prominent, likely due to the fact that the 100,000 circuit structures represent only a finite subset of the possibilities for 12-qubit circuit. As a result, the number of circuits that can be learned is limited. Most high-performance circuits are distributed along the left edge of the latent space, with a color gradient transitioning from dark to light as one moves from right to left.

Compared with subfigures 2(i), 2(j), 2(k), 2(l), 2(m), and 2(k), which utilize the encoding scheme $\mathcal{E}^{GSQAS}$ and show more loosely distributed red points, our new encoding results in a more concentrated and smoother latent representation, as demonstrated in subfigures 2(a), 2(b) and 2(c).

Figure: (a) PCA4 QC${}_{H_{2}}$
Refer to caption: extracted/5949113/images/vqe-model-circuits_4_qubits_quantum_arch2vec_full_embedding_full_embedding_smooth.png

### 4.2 Quantum Architecture Search (QAS) Performance

Observation (1): In Figure 3, we present the average reward per 100 searches for each experiment. The results show that both the REINFORCE and BO methods effectively learn to navigate the latent representation, leading to noticeable improvements in average reward during the early stages. In contrast, Random Search fails to achieve similar improvements. Furthermore, although the plots indicate slightly higher variance in the average reward for the REINFORCE and BO methods compared to Random Search, their overall average reward is significantly higher than that of Random Search.

Figure: (a) State preparation
Refer to caption: extracted/5949113/images/4-qubits-fidelity_avg_reward_per_100_with_var_filling.png

Observation (2): In Figure 4, we illustrate the number of candidate circuits found to achieve a preset threshold after performing 1000 searches using the three search methods. The results show that the 8-qubit experiments are more complex, resulting in fewer circuits meeting the requirements within the search space. Additionally, within a limited number of search iterations, both the REINFORCE and BO methods are able to discover a greater number of candidate circuits that meet the threshold, even in the worst case, i.e., when comparing the minimal number of candidates. Notably, their performance significantly surpasses that of the Random Search method, especially REINFORCE, despite the fact that the difference between the minimal and maximal number of candidates indicates that REINFORCE is more sensitive to the initial conditions compared to the other two approaches. These findings highlight the clear improvements and advantages introduced by QAS based on the latent representation, which enables the efficient discovery of numerous high-performance candidate circuits while reducing the number of searches required.

Figure: (a) 4-qubit experiments
Refer to caption: extracted/5949113/images/4-qubits_experiments_candidates.png

**Table 2: Compare the QAS performance of different QAS methods for the 4-qubit tasks. URL denotes unsupervised representation learning, $F_{thr}$ is the threshold to filter poor-performance architectures, $N_{lbl}$, $N_{rest}$ and $N_{>0.95}$ refer to the number of required labeled circuits, rest circuits after filtering and the circuits that achieve the performance higher than 0.95 in the rest circuits respectively. $N_{eval}$ represents the number of evaluated circuits, i.e. the sum of the number of labeled and sampled circuits, $N_{QAS}$ is the number of searched candidates in average of 50 runs.**
| Method | $Task$ | $F_{thr}$ | $N_{lbl}$ | $N_{rest}$ | $N_{>0.95}$ | $N_{eval}$ | $N_{QAS}$ | $N_{QAS}/N_{eval}$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| GNN^URL | Fidelity | 0.5 | 1000 | 21683 | 780 | 2000 | 36 | 0.0180 |
| Max-Cut | 0.9 | 1000 | 45960 | 35967 | 2000 | 783 | 0.3915 |  |
| QC-4${}_{H_{2}}$ | 0.8 | 1000 | 65598 | 18476 | 2000 | 278 | 0.1390 |  |
| GSQAS^URL | Fidelity | 0.5 | 1000 | 21014 | 768 | 2000 | 37 | 0.0185 |
| Max-Cut | 0.9 | 1000 | 43027 | 33686 | 2000 | 785 | 0.3925 |  |
| QC-4${}_{H_{2}}$ | 0.8 | 1000 | 30269 | 19889 | 2000 | 658 | 0.3290 |  |
| Random Search | Fidelity | - | 0 | 100000 | 1606 | 1000 | 15 | 0.0150 |
| Max-Cut | - | 0 | 100000 | 57116 | 1000 | 568 | 0.5680 |  |
| QC-4${}_{H_{2}}$ | - | 0 | 100000 | 37799 | 1000 | 371 | 0.3710 |  |
| QAS${}^{URL}_{RL(BO)}$ | Fidelity | - | 0 | 100000 | 1606 | 1000 | 69(63) | 0.0690(0.0630) |
| Max-Cut | - | 0 | 100000 | 57116 | 1000 | 898(820) | 0.8980(0.8200) |  |
| QC-4${}_{H_{2}}$ | - | 0 | 100000 | 37799 | 1000 | 817(739) | 0.8170(0.7390) |  |

**Table 3: We compare the QAS performance of different encodings using various search methods. For the 4- and 12-qubit quantum chemistry tasks, we select $H_{2}$ and $LiH$, respectively, while for the 8-qubit task, we use the TFIM. The results represent the average of 50 runs.**
| Method | Encoding $\mathcal{E}$ | $N_{rest}$ | $N_{eval}$ | $N_{QAS}$ | $N_{QAS}/N_{eval}$ |
| --- | --- | --- | --- | --- | --- |
| GSQAS4 | GSQAS | 25996 | 2000 | 625 | 0.3125 |
| Ours | 30269 | 2000 | 658 | 0.3290 |  |
| GSQAS12 | GSQAS | 60088 | 2000 | 283 | 0.1415 |
| Ours | 60565 | 2000 | 276 | 0.1380 |  |
| QASRL-4 | GSQAS | 100000 | 1000 | 760 | 0.7600 |
| Ours | 100000 | 1000 | 817 | 0.8170 |  |
| QASRL-8 | GSQAS | 100000 | 1000 | 160 | 0.1600 |
| Ours | 100000 | 1000 | 167 | 0.1670 |  |
| QASRL-12 | GSQAS | 100000 | 1000 | 422 | 0.4220 |
| Ours | 100000 | 1000 | 392 | 0.3920 |  |

Observation (3): In Table 2, we compare various QAS methods with our approach on the 4-qubit state preparation task, using a circuit space of 100,000 circuits and limiting the search to 1000 queries. GNN^URL and GSQAS^URL represent predictor-based methods from He et al. () and He et al. (), respectively, both employing our pre-trained model. QAS${}^{URL}_{RL(BO)}$ denotes the QAS approach with REINFORCE (BO) used in this work. The average results over 50 runs indicate that both the predictor-based methods and our approach are capable of identifying a significant number of high-performance circuits with fewer samples.
However, predictor-based methods rely on labeled circuits to train predictors, introducing uncertainty as they may inadvertently filter out well-performing architectures along with poor ones. While a higher $F_{thr}$ value filters out more low-performance circuits, increasing the proportion of good architectures in the filtered space, it also sacrifices many well-performing circuits, which can lead to improved Random Search performance but at the cost of excluding some optimal circuits.
Despite these trade-offs, our method achieves comparable performance to predictor-based methods, demonstrating higher efficiency in terms of $N_{QAS}/N_{eval}$ while requiring fewer circuit evaluations.
In Appendix A.4, we present the best candidate circuits acquired by each of the three methods for every experiment.

Observation (4): In Table 3, we present the search performance across different frameworks and encoding methods, focusing on 4-, 8-, and 12-qubit quantum chemistry tasks for comparison. In most cases, our encoding method achieves the highest search efficiency, although the performance for the 12-qubit task is slightly lower than with another encoding method. Combined with the representation learning results in Figure 2, we observe that the search is significantly more efficient when the learned circuit representation is smooth and concentrated. For the 12-qubit experiments, the circuits used for representation learning may be insufficient to fully capture the search space, leading to representation learning failures, as shown in Figure 2(d), and resulting in a decline in search efficiency.

## 5 Conclusion

Inspired by the Arch2vec method (Yan et al., ), we focus on exploring whether unsupervised architecture representation learning can enhance QAS. By decoupling unsupervised architecture representation learning from the QAS process, we successfully eliminate the need for a large number of labeled circuits. Additionally, our proposed quantum circuit encoding scheme addresses limitations in existing representations, improving search performance through more accurate and effective embeddings. Furthermore, our framework conducts QAS without relying on a predictor by directly applying search algorithms, such as REINFORCE and Bayesian Optimization (BO), to the latent representations. We have demonstrated the effectiveness of this approach through various experiments. In our framework, the success of QAS depends on the quality of unsupervised architecture representation learning and the selection of search algorithms. Thus, we recommend further investigation into architecture representation learning for QAS, as well as the development of more efficient search strategies within the latent representation space.

#### Acknowledgments

The project of this workshop paper is supported with funds from the German Federal Ministry of Education and Research in the funding program Quantum Reinforcement Learning for industrial Applications (QLindA) - under project number 13N15644 and the Federal Ministry for Economic Affairs and Climate Action in the funding program Quantum-Classical Hybrid Optimization Algorithms for Logistics and Production Line Management (QCHALLenge) - under project number 01MQ22008B. The sole responsibility for the paper’s contents lies with the authors.

## Appendix A Appendix

### A.1 Circuit Generator Settings

The predefined operation pool which defines allowed gates in circuits is important for QAS as well, because a terrible operation pool such as one with no rotation gates or no control gates cannot generate numerous quantum circuits with excellent expressibility and entanglement capability. This makes the initial quantum search space poor, so it will influence our further pre-training and QAS process. Therefore, we choose some generally used quantum gates in PQCs as our operation pool $\{\texttt{X, Y, Z, H, Rx, Ry, Rz, U3, CNOT, CZ, CY}\}$ for the circuit generator to guarantee the generality of our quantum circuit space. Other settings of the circuit generator are summarized below:

**Table 4: Description of settings predefined for the circuit generator.**
| Hyperparameter | Hyperparameter explanation | Value for 4/8/12-qubit experiments |
| --- | --- | --- |
| num-qubits | the number of qubits | 4/8/12 |
| num-gates | the number of gates in a circuit | 10/20/30 |
| max-depth | the maximal depth in a circuit | 5 |
| num-circuits | required the number of circuits | $10^{5}$ |

### A.2 Application Settings

Figure: (a) The target circuit of the 4-qubit state preparation
Refer to caption: extracted/5949113/images/4-qubits-fidelity_target.png

##### Quantum State Preparation.

In quantum information theory, fidelity (Liang et al., ) is an important metric to measure the similarity of two quantum states. By introducing fidelity as the performance index, we aim to maximize the similarity of the final state density operator with a certain desired target state. We first obtain the target state by randomly generating a corresponding circuit, and then with a limited number of sample circuits, we use the search methods to search candidate circuits that can achieve a fidelity higher than a certain threshold. During the search process, the fidelity can be directly used as a normalized reward function since its range is [0, 1]. Figure 5 shows the circuits used to generate the corresponding target states.

##### Max-cut Problems.

The max-cut problem (Poljak & Rendl, ) consists of finding a decomposition of a weighted undirected graph into two parts (not necessarily equal size) such that the sum of the weights on the edges between the parts is maximum. Over these years, the max-cut problem can be efficiently solved with quantum algorithms such as QAOA (Villalba-Diez et al., ) and VQE (using eigenvalues). In our work, we address the problem by deriving the Hamiltonian of the graph and using VQE to solve it. We use a simple graph with the ground state energy $-10\ Ha$ for the 4-qubit experiment and a relatively complex graph with the ground state energy $-52\ Ha$ in the case of the 8-qubit experiment. Furthermore, we convert the energy into a normalized reward function integral to the search process. The visual representations of these graphs are presented below:

Figure: (a) The 4-qubit max-cut graph
Refer to caption: extracted/5949113/images/4-qubit-max_cut.png

##### Quantum Chemistry.

In the field of QC, VQE (Peruzzo et al., ; Tilly et al., ) is a hybrid quantum algorithm for quantum chemistry, quantum simulations, and optimization problems. It is used to compute the ground state energy of a Hamiltonian based on the variational principle. For the 4- and 12-qubit quantum chemistry experiment, we use the Hamiltonian of the molecule $H_{2}$ and $LiH$ and its common approximate ground state energy $-1.136\ Ha$ and $-7.88\ Ha$ as the optimal energy. As for the 8-qubit experiment, we consider $n=8$ transverse field Ising model (TFIM) with the Hamiltonian as follows:

$$ $\displaystyle{\bm{H}}=\sum_{i=0}^{7}\sigma_{z}^{i}\sigma_{z}^{(i+1)\ mod\ 6}+ \sigma_{x}^{i}\text{.}$ (6) $$

We design some circuits to evaluate the ground state energy of the above Hamiltonian and get an approximate value $-10\ Ha$ as the optimal energy. According to the approximate ground state energy, we can use our methods to search candidate circuits that can achieve the energy reaching a specific threshold. In the process of searching for candidates, the energy is normalized as a reward function with the range [0, 1] to guarantee search stability.

### A.3 Hyperparameters of Pre-training

Table 5 shows the hyperparameter settings of the pre-training model for 4-qubit and 8-qubit experiments.

**Table 5: Description of hyperparameters adopted for pre-training.**
| Hyperparameter | Hyperparameter explanation | Value for 4/8/12-qubit experiments |
| --- | --- | --- |
| bs | batch size | 32 |
| epochs | traning epochs | 16 |
| dropout | decoder implicit regularization | 0.1 |
| normalize | input normalization | True |
| input-dim | input dimension | 2+$\#$gates+$\#$qubits |
| hidden-dim | dimension of hidden layer | 128 |
| dim | dimension of latent space | 16 |
| hops | the number of GIN layers ($L$ in eq.2) | 5 |
| mlps | the number of MLP layers | 2 |

### A.4 Best Candidate circuits

Observation (5): In Appendix A.4, we present the best candidate circuits acquired by each of the three methods for every experiment. These circuits exhibit a higher likelihood of being discovered by REINFORCE and BO in contrast to Random Search. This observation underscores the superior search capabilities of REINFORCE and BO in navigating the large and diverse search space generated by our approach, which is based on a random generator derived from a fixed operation pool. Unlike conventional approaches that adhere to layer-wise circuit design baselines, our method excels in discovering circuits with fewer trainable parameters. This characteristic is of paramount importance when addressing real-world optimization challenges in QAS. In conclusion, our approach not only enhances the efficiency of candidate circuit discovery but also accommodates the distinct characteristics of various problem domains through a large and diverse search space.

Figure: (a) 4-qubit state preparation
Refer to caption: extracted/5949113/images/best_candidate/4-qubits-fidelity_best_candidate_77132.png

## Peer Reviews

confidence: 5
contribution: 1
initial_rating: 3
presentation: 2
questions: No questions
soundness: 1
strengths: Well-structured paper.
summary: The authors focused on automatically designing the quantum circuit to reach acceptable accuracy while keeping the circuit at a very low circuit depth. They tried to utilize a representation learning model to evaluate the quantum circuits generated by Reinforcement Learning and Bayesian Optimization. Numerical results are provided on state preparation, Max-cut, and Ground-state energy estimation problems with the comparison to a AAAI paper and random search.
weaknesses: 1. First, the authors should pay more attention to the background check of the related works in the quantum architecture search area.  The conclusion in section 2 that "these methods require the evaluation of numerous quantum circuits" is indeed incorrect for the cited paper. Recent QAS approaches can generate circuit architecture while optimizing the internal parameters (you can refer to the survey [1]), which does not require evaluating the generated circuits. 
2. The most important concern is the motivation. The paper is titled "Quantum Architecture Search...," but it is indeed about representation learning of a quantum circuit. The search methods proposed in this paper were not designed by the authors. The authors tried to utilize the learned representation to evaluate the generated quantum circuit, which I find quite strange. The representation model should be trained with a certain qubit number and a certain Hamiltonian, indicating the proposed method is not scalable or generalizable. If you adopt recent models on the QAS, you can generate the quantum ansatz with optimized parameters, which can be easily evaluated with a simple evaluation of the output state with the given Hamiltonian or target state. I don't see the need to utilize a representation learning model here. 
3. The evaluation metric can not reflect the actual performance of the proposed methods. The ground-state estimation for quantum chemistry simulation requires admissible results within chemical accuracy (1.6 mHa) [3]. Generating a circuit with a maximum depth of 5 layers is too naive for the tasks. The only baseline method is from [2], which is weak considering the fact that there are numerous other QAS methods. 







[1] Quantum circuit synthesis and compilation optimization: Overview and prospects

[2] Experimental quantum computational chemistry with optimized unitary coupled cluster ansatz

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. How to make sure in the decoder output, the generated adjacent matrix and the gate matrix match with each other.
2. Why use GAE and VGAE as baseline, they are pretty old models.
3. Theoretically, it is very hard to efficiently represent an arbitrary quantum circuit ansatz without a exponential scaling size of vector. Do you really need arbitrary ansatz? Some ansatz block like efficientSU2 are comprehensive enough to represent arbitrary unitary, why we don't use them as building blocks? Conceptually, it is similar case in classical NAS, we don't search for arbitrary neural architecture, instead, we search with building blocks such as conv, attention, MLP etc.
soundness: 2
strengths: 1. The paper tries to decouple the representation learning of the quantum circuit to the search process, which is an important direction towards more general search of variational quantum ansatz.
2. The paper introduces a refined encoding with novel gate matrix and adjacency matrix that gets the detailed circuit characteristics, control and target qubit positions in multi-qubit gates. This scheme benefits downstream tasks by enhancing the structural understanding to the circuit.
summary: The paper propose a framework for Quantum Architecture Search (QAS) utilizing unsupervised representation learning to optimize quantum circuits for Variational Quantum Algorithms (VQAs). The framework is inspired by the Arch2vec algorithm for classical neural architectures and it decouples representation learning from search, and enables efficient latent-space exploration without labeled datasets. It proposes an improved quantum circuit encoding scheme that refines gate connectivity representations and a variational graph isomorphism autoencoder (GIN) for encoding circuit structures. The QAS search process leverages REINFORCE-based reinforcement learning and Bayesian optimization (BO), and shows efficiency improvements in identifying high-performing circuits for quantum state preparation, max-cut problem, and quantum chemistry applications.
weaknesses: 1. The efforts to find a way that decouples the circuit ansatz representation to the search process is important. However, the proposed method is not a scalable way to do it. For small size circuit, it is easy for the encoder and decoder to encode the circuit with a compact vector. However, the paper only demonstrated the number of qubit fewer than 12. and the reconstruction accuracy of 12 qubits is only 98.76%. The accuracy of reconstruction will degrades significantly when the number of qubit increases, as the design space increases exponentially.
2. Beside the difficulty to encode circuit with large number of qubit, the data required to train the encode and decoder is also increasing exponentially to cover the space. 
3. For large number of qubits, to achieve arbitrary unitary, the required number of 2 q gate increases exponentially, the size of adjacent matrix and gate matrix also increase exponentially.
The author should justify the effectiveness and scalability to a reasonable size, for example, 50 qubits. Because a 12 qubit VQA will not generate any quantum advantage.

minor
1. Line 290, the "ops", "qubit", "adj", "mean" should be underscore
2. The text is too small to read in figure 4

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. Could the authors specify the adaptations made in the encoder and decoder specifically for the QAS task? While the paper describes the overall model structure including the GINs and RLs, it would be helpful to clarify how these changes, such as the addition of a fusion layer after the GIN, influence the learning and final performance. Currently, the innovation in this area feels somewhat ambiguous, and further detail on how the QAS-related adaptation contributes to performance would be insightful. 

2. The scalability of this approach is not fully demonstrated in the current experiments, as the main results are based on circuits with up to 12 qubits (mainly over 4 and 8 qubits) and a maximum depth of 5. It would be beneficial to include experiments on larger qubit circuits and deeper circuits to better assess the model’s scalability and robustness in more complex settings.

3. It appears that only two baselines are used for comparison. Expanding the number of baseline methods would provide a clearer picture of the model’s relative performance and robustness, enhancing the strength of the comparisons.

4. I suggest reorganizing the paper structure to enhance readability. For instance, consolidating hyperparameter values in the experimental section, rather than embedding them within the methodology, would create a more streamlined and accessible flow for readers.

5. In addition to 1, an ablation study of important modules may help the demonstration.
soundness: 2
strengths: 1.	The paper introduces a novel approach to Quantum Architecture Search (QAS) that leverages unsupervised representation learning, eliminating the need for labeled datasets and predictors. 
2.	The authors propose an improved quantum circuit encoding scheme, representing circuits as directed acyclic graphs (DAGs) with specific positional information for two-qubit gates. 
3. The paper demonstrates the versatility of its framework by employing both reinforcement learning (REINFORCE) and Bayesian optimization (BO) as search strategies within the learned latent space.
summary: This paper, Quantum Architecture Search with Unsupervised Representation Learning, presents a novel approach to optimizing quantum circuit architectures for variational quantum algorithms on NISQ devices. It introduces a predictor-free Quantum Architecture Search (QAS) framework that uses unsupervised representation learning to eliminate the need for labeled data. The authors propose an improved encoding scheme for representing quantum circuits as directed acyclic graphs (DAGs) and use a variational graph isomorphism autoencoder (GIN) to learn smooth latent representations of circuit architectures. To explore this representation space, they apply reinforcement learning and Bayesian optimization, demonstrating the framework’s effectiveness across quantum machine learning tasks like quantum state preparation, max-cut, and quantum chemistry.
weaknesses: See the questions listed below.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: - How scalable is the representation? For realistic circuits that could potentially offer some advantage, e.g. with hundreds of qubits and depth close to linear, are these graphs learnable?
soundness: 2
strengths: - The code is open source which is beneficial for reproducibility
- Figure 1 and the explanation is helpful
- The latent approach to QAS to avoid expensive repeated circuit evaluations during every search is interesting and warrants further research as an idea
summary: This paper presents an algorithm for quantum architecture search based on the arch2vec algorithm. Specifically, they generate quantum circuits and learn a latent representation for their structure using a graph based autoencoder, then optimize generated circuits based on the learned latent representation.
weaknesses: - The empirical justifications are lacking. Given that circuit width and depth will be much larger than the circuits shown, strong arguments are necessary in regards to scalability. However, existing results seem to indicate pretty negative correlations with scale. Figures 3 and 4 show that 8 qubits achieves much lower performance than the 4 qubit. The scale is also limited when it comes to simulations, given that up to 20 qubits is doable on laptops. There is no compelling arguments for how these methods would perform under practically interesting/relevant conditions and this needs to be included.
- Subjective statements could be quantified: e.g. “show more loosely distributed red points, our new encoding results in a more concentrated and smoother latent representation” this could be analyzed rather than the just claimed visually
- Table 1 shows the results of the pretraining for the latent space, which would benefit from uncertainty estimates (since these are dependent, ideally minimally, on the initializations/random keys)
- It would be worth showing other algorithms as well (even if in the appendix) that are mentioned in the text. Specifically, PPO and A2C are mentioned as algorithms that were evaluated but didn’t perform as well as REINFORCE, having plots showing that in the appendix would be worthwhile
- Minor graphical points: e.g. coloring on figure 3(c) left is wrong for random search
- Minor grammatical points: e.g. “We are considering using” → “we use”