TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search
Authors: Akash Kundu
Stefano Mangini
Venue: NeurIPS 2025 poster ()
Keywords: quantum architecture search
tensor network
reinforcement learning
Primary area: machine_learning_for_sciences
TL;DR: We introduce TensorRL-QAS, an improved framework for quantum architecture search that combines tensor network initialization with reinforcement learning.

Abstract: Variational quantum algorithms hold the promise to address meaningful quantum problems already on noisy intermediate-scale quantum hardware. In spite of the promise, they face the challenge of designing quantum circuits that both solve the target problem and comply with device limitations. Quantum architecture search (QAS) automates the design process of quantum circuits, with reinforcement learning (RL) emerging as a promising approach. Yet, RL-based QAS methods encounter significant scalability issues, as computational and training costs grow rapidly with the number of qubits, circuit depth, and hardware noise. To address these challenges, we introduce TensorRL-QAS, an improved framework that combines tensor network methods with RL for QAS. By warm-starting the QAS with a matrix product state approximation of the target solution, TensorRL-QAS effectively narrows the search space to physically meaningful circuits and accelerates the convergence to the desired solution. Tested on several quantum chemistry problems of up to 12-qubit, TensorRL-QAS achieves up to a 10-fold reduction in CNOT count and circuit depth compared to baseline methods, while maintaining or surpassing chemical accuracy. It reduces classical optimizer function evaluation by up to 100-fold, accelerates training episodes by up to 98\%, and can achieve 50\% success probability for 10-qubit systems, far exceeding the $<$1\% rates of baseline. Robustness and versatility are demonstrated both in the noiseless and noisy scenarios, where we report a simulation of an 8-qubit system. Furthermore, TensorRL-QAS demonstrates effectiveness on systems on 20-qubit quantum systems, positioning it as a state-of-the-art quantum circuit discovery framework for near-term hardware and beyond.

TensorRL-QAS: Reinforcement learning with tensor
networks for improved quantum architecture search
Akash Kundu
Stefano Mangini
QTF Centre of Excellence Department of Physics University of Helsinki, Finland akash. kundu@helsinkk
QTF Centre of Excellence Department of Physics
University of Helsinki, Finland, Algorithmiq atefano..anginni wwt.ginnn@elssiii...
Abstract
Variational quantum algorithms hold the promise address meaningful quan- tum problems already on noisy intermediate- scale quantum hardware. In spite of the promise, they face the challenge of designing quantum circuits that both solve the target problem and comply with device limitations. Quantum architecture search (QAS) automates the design process of quantum circuits, with reinforcement learning (RI.) emerging as promising approach. Yet, RL-based QAS methods encounter significant scalability issues, as computational and training costs grow rapidly with the number of qubits, circuit depth, and hardware noise. To address these challenges, introduce TensorRL- QAS, an improved framework that com- bines tensor network methods with RL for QAS. By warm-starting the QAS with a matrix product state approximation of the target solution, TensorRL-QAS effec- tively narrows the search space to physically meaningful circuits and accelerates the convergence to the desired solution. Tested on several quantum chemistry problems of up to 12-qubit, TensorRL- CNOT count and circuit depth compared to baseline methods, while maintaining or surpassing chemical accuracy. It reduces classical optimizer function evaluation by up to 100-fold, accelerates training episodes by up to 98%, and can achieve 50% success probability for 10-qubit systems, far exceeding the <1% rates of baseline. Robustness and versatility are demonstrated both in the noiseless and noisy scenarios, where report simulation of an 8-qubit system. Furthermore, TensorRL-QAS demonstrates effectiveness on systems on 20-qubit quantum sys- tems, positioning it as a state-of-the-- art quantum circuit discovery framework for near-term hardware and beyond.
achieves up to 10-fold reduction in
Introduction
Despite the promise quantum computing in recent years, the practical realization quantum advantage remains elusive, primarily due the limitations of current noisy intermediate-scne quantum (NISQ) hardware [1]. These devices are characterized by restricted qubit counts, limited connectivity, and significant noise, severely constraining the complexity of quantum circuits that can be reliably executed. As result, most foundational quantum algorithms are not yet feasible on underscoring the urgent need for novel algorithmic strategies that are both effective and frriware-friendll.
available quantum hardware [
To bridge this gap, hybrid quantum-classical approaches have emerged practical solution. Among these, variational quantum algorithms (VQAs) have become the leading paradigm for NISQ VQAs operate by iteratively optimizing the parameters of a parameterized
devices 3
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Figure 1: Schematic representation of TensorRL-QAS algorithm. Given Hamiltonian for which we seek the lowest eigenstate. By combining tensor network (TN) with reinforcement learning (RL)-based quantum architecture search (QAS), we find solutions that would not be achievable using either approach alone. Inspired by [12), we obtain an approximate ground state of the Hamiltonian using DMRG [13 14], then use this result warm-start the QAS training in RL-framework. We
employ Riemannian optimization on the Stiefel manifold to map the MPS obtained from DMRG to a
quantum circuit
15
1) Use DMRG flnd approximate ground state
2) Map MPS quantum circuit via Riemannian optimization
Improve solution with reinforcement learning (RL) based quantum architeture search (QAS)
Oberrve state megg -)
Rl, agent
arg min 99999
U/
palcy
(00)
(19)
Add new gate
Marris Priciuu Stote (MPS)
1. G [1rad]1199/100
Optimine eircuit pararacters (CPT) (o(0)) - (w(0r))
quantum circuit (PQC) to minimize problem-speee -ppcific cost function, typically the expectation value of a Hamiltonian H. In this framework, a quantum state is prepared using a PQC, (0). the cost is measured, and the parameters @ are updated via a classical optimizer to minimize the cost. The effectiveness of VQAs heavily depends on the choice of the ansatz for the underlying PQCs. Typically, PQC architectures are fixed prior to optimization, often guided by hardware connectivity 7 or by problem-specine strategies have shown early promise, the resulting circuits are often either too shallow to be expressive or too deep to noise-resilient, thus limiting the scalability and performance of VQAs 55 6 [1].
function C(0)= = (ojUi(
insights
B g 10. While these
To address these challenges, the field has recently turned its attention to quantum architecture search (QAS) [16 17 18) which seeks to automate the discovery of optimal PQC structures from a finite pool of quantum gates. The goal of QAS is to identify an arrangement of gates and corresponding parameters that minimize the target cost function, thereby tailoring circuit architectures to both the problem at hand and the specific constraints of the quantum hardware. Among various QAS strategies, reinforcement learning (RL) has emerged as a particularly promising approach for navigating the vast and complex space of possible circuit architectures [19 20, [21 [2].. In this framework, the RL-agent sequentially constructs PQCs by selecting gates and their placements, using feedback from the quantum cost function as a reward signal to guide policy updates. Despite the significant potential of RL-bbased QAS methods, their scalability remains a critical bottleneck. To date, most RL-drrven QAS algorithms have only been demonstrated on quantum problems involving up to 8-qubit in noiseless simulations and up to 4-qubit in realistic noisy scenarios. The primary obstacles to scaling include the rapid growth of the action space with increasing qubit number and the need for longer episodes, which together demand a prohibitive number of queries to quantum simulators, let alone real quantum devices. Moreover, the computational cost of simulating quantum circuits grows rapidly with circuit depth, making QAS for larger systems exceedingly challenging (see Appendix [L for a detailed discussion).
Our work is motivated by the critical need to overcome scalability limitations in RL-baaed QAS. To address the dual challenges of action space explosion and prohibitive simulation costs, we introduce TensorRL-QAS, a novel framework that combines reinforcement learning with tensor network (TN) methods for QAS. At its core, TensorRL-QAS employs problem-aware initialization obtained with TN methods, strategically initializing the search space to a physically meaningful starting point 12 23
Tensor network methods have become a cornerstone for scalable simulations of quantum systems, providing compact representations of complex, high-dimensional quantum states by means of an efficient representation of entanglement (24 25 Their versatility and efficacy have been demon-
strated in several tasks, like simulating utility-scale quantum computations [26] 27 characterizing quantum states and operations
28 29
30, and for quantum error mitigation 31 32 Following the
idea originally proposed in 22
23 on tensor network pre-training of quantum circuits, in this work.

---

we investigate the integration of TNs and RL by warm-starting an RL-based QAS procedure with TN methods. We schematically summarize the main idea of the manuscript in Fig. I
Given a Hamiltonian for which we seek a circuit representation of the ground state, we first use DMRG L [4] find matrix product state (MPS) approximation of said ground state, then use Riemannian optimization tools to map such MPS to a quantum circuit [15), and eventually refine the circuit using RL-QAS, In particular, we test two different approaches to enhance scaling and reduce the computational cost in RL-QAS: (1) TensorRL (trainable TN-init), in which the TN warm-start is part of the RL-state, and the agent can modify its parameters while training: whereas in (2) TensorRL (fixed TN-init), the TN warm--star is fixed and not part of the RL-state, so the agent cannot modify its parameters. To assess the efficacy of these methods, we compare them against a baseline, where the TN warm-start is replaced with a circuit having the same structure but zeroed-out parameters, namely StructureRL. While not a method of independent interest, it enables direct comparison and demonstrates convergence with the other approaches, given sufficient training.
Through extensive benchmarks on molecular ground-state problems up to 12-qubit, both in noiseless and noisy regimes, TensorRL-QAS consistently produces more compact circuits, while maintaining or surpassing chemical accuracy, and shortens execution times, for example, reducing function evaluations by classical optimizer up to 100-fold and accelerating training episodes by up to 98% compared baseline QAS methods. By integrating well-establisher TN methods into RL-based QAS, our methodology enables an improved exploration of PQC architectures, reducing the gap to a practical upplication of VQAs on NISQ hardware and beyond.
2
Related works
Reinforcement learning (RL) has become a prominent framework for optimizing parameterized quantum circuits (PQCs) in variational quantum algorithms, where agents are trained via tailored reward functions to select effective quantum gate sequences. Approaches such as double-deep Q-networks (DDQN) with e-greedy policies have been used for ground state estimation [2I], while actor-critic and proximal policy optimization methods have enabled the construction of multi-qubit entangled states [33] [3]. Deep RL has also facilitated hardware- aware circuit design and improved efficiency (20). with recent work incorporating curriculum learning, pruning (35] for quantum architecture search (QAS), and tensor decomposition for T-count minimization 33 In the scope of benchmarking RL-based QAS, ref. (37] introduces random agent (RA) QAS where the agent chooses the action at each step randomly from a uniform distribution in the RL-framework. Despite these advances, RL-based methods are often constrained circuits with only few qubits due the exponential growth of the action space and the high computational cost per episode.
Alternative -L--Red strategies include sampling-based methods for circuit selection in ground state estimation [6. 17 Monte Carlo sampling for PQC discovery in QFT and Max-Cur [18), and supernet-bad based weight sharing for quantum chemistry tasks (38). Simulated annealing has also been explored for PQC optimization [39]. Mcanwhile, evolutionary algorithms such as EQAS-PQC [40] have been introduced to construct PQCs more effectively. Differentiable search techniques such as quantumDART'S 41 leverage Gumbel-Softtax sampling for variational algorithms. In ref. [42), the author introduces path- and expressivity- based training- QAS. Neural predictor-basee based approaches [43) have also been proposed, where a neural network guides quantum architecture search by predicting circuit performance, enabling more efficient exploration of the search space. While these approaches expand the landscape of quantum circuit optimization, scalability and robustness under noise remain open challenges in both noisy and noiseless settings. For a comprehensive review
of QAS, we refer the interested readers to 44 45 46
3 Methods
In the same spirit of [12, in this work, we use a tensor network (TN) warm-start the RL-based quantum architecture search (QAS) (see Appendix (A a discussion on how our approach differs from previous TN and VQA synergetic methods). Specifically, we focus our investigation on problems where the goal is to find a parameterized quantum circuit (PQC) that approximately prepares the ground state of a given Hamiltonian of interest. Our approach consists of three main steps, illustrated in Fig. I Given target Hamiltonian H. the procedure goes as follows
3

---

1. Use density matrix renormalization group (DMRG) IE 14 with a maximum allowed bond dimension X to find a matrix product state (MPS) v approximation of the ground state of H.
2. Map such MPS to a PQC ( + U (0) using efficient TN contractions and Riemannian optimization on the Stiefel manifold [T] 47 48).
3. Continue with RL-based QAS to add more gates to the circuit U (0) - VU (0) to further minimize the energy and thus better approximate the desired ground state.
Clearly, if DMRG alone accurately obtains the desired ground state, subsequent steps unnecessary. However, if it fails at maximum bond dimension X-a parameter that governs both the computational cost and the expressiveness of the underlying tensor network model- the solution can be improved via access to a quantum computer combined with the RL-QAS framework.
3.1 DMRG and MPS
Density matrix renormalization group (DMRG) I3
44 49 50
a well-established and arguably
the most used procedure for finding ground states of quantum chemistry Hamiltonians [5]] It does so by solving the optimization problem
V
arg min (b)H)))
(1)
|0 #1l=1
where (15) is the many-body wavefunction, and the solution [V) is a state modelled by a so-called matrix product state (MPS), which is a classically efficient parameterization of high nigh-dimensional
vectors. MPS are also known as tensor trains in numerical linear algebra literature [52] 533. An MPS for system of N-qubit can be written explicitly as
(V) Σ
A! A ... (M) li1éz iN),
[k] E C E C x 0%,
E [1,N}
(2)
k & [1,
where A!ll and A are row- and column-vectors, respectively, and the rest are matrices. The shapes are such that the whole vector-matrices-vecte these have size at most da S x N. The hyperparameter X called bond dimension, and controls the amount of correlations (entanglement, in the case of quantum) that the MPS can represent. The larger the bond dimension, the more expressive the model is, and thus is capable of representing more general vectors. Briefly, DMRG works by iteratively sweeping through each site in the MPS and optimizing the corresponding local matrix to minimize the cost in ① This sweeping continued until convergence, with the computational cost of the whole procedure scaling linearly with the number of sites and polynomially with the bond dimension X- An in-depth explanation of DMRG and the MPS ansatz is outside the scope of this work, we refer the interested reader to
multiplication is well-defined. Importantly, all of
[1 49 50)
3.2 Mapping MPS to PQC via Riemannian optimization
Given the approximate solution ( in MPS form, we proceed by finding a quantum circuit that prepares V). That is, we seek a unitary U such that U (0) VV). Several proposals have been
put forward to address this task, some including explicit constructions [24] 54 551, others based on
58 59]. In this work, adopt the second approach due to its hardware efficiency, as it does not require ancillary qubits and densely connected topologies, and for its better empirical performance [56] 57). Furthermore, as the PQC will be further modified by the following RL-based QAS procedure, an approximate, rather than exact, reconstruction suffices. We consider unitaries U which can be decomposed as product of 2-qubit unitaries, namely U IT U2 with Un E U(4). This is because most quantum computers natively implement 2-qubit interactions, and so the circuit can be mapped to real devices with minimum overhead. The actual form of the unitary U can be chosen freely, but in practice it should satisfy some requirements, like matching the topology of the quantum hardware and reducing the number of operations and depth to minimize the detrimental effect of noisy gates. In this manuscript, we focus unitaries with brickwork structure, as illustrated in Fig. [ This hardware-efficient ansatz particularly suited for devices with linear connectivity 77. though alternative designs could also be considered.
iterative variational approaches [56] 57

---

The circuit can be found by maximizing the overlap between the target state ()) and the variational model, that is maximizing the loss function
U1
L(U,, (2..,U...... KV =
(M))
Ui
(3)
A=
Up
E we show an example of TN representing the overlap. The minimization process is done by first constructing the overlap TN and then using automatic differentiation to compute the gradients and update all unitaries simultaneously. The contraction of the tensor network and the computation of the gradients remain efficient as long as the PQC is shallow enough so that the X in the contraction remains feasible.
whose trainable parameters are the 2-qubit unitaries {Ut}. In Eq.
Fundamentally, the brased update rule preserves the unitarity of the trainable matrices. This can be obtained by employing Riemannian optimization techniques on the Stiefel manifold [47] 48, [5]. At a high level, the procedure works by modifying the (Euclidean) gradients so that the new updated matrices live on the same manifold as the starting ones. This can be done in several ways, for example, using SVD [56] or, as we instead do, with the so-called Cayley transform [47), which was shown to yield better optimization performances in many quantum-relan relatted optimization tasks [15]. Following [5]] we have adapted Adam [60] to perform Riemannian optimization on the Stiefel manifold and use to minimize the loss in Eq. 3 We refer to Appendix B for an extended discussion and technical derivations, including the full algorithm for Riemannian optimization.
Appendix H.6 provides detailed benchmarking of how the number of layers in the brickwork affects training performance when approximating ground-state energies across several molecular systems. Moreover, Appendix H.6 demonstrates that the contribution of the MPS-to-circuit mapping in ground-state preparation and that advanced optimization methods (e.g., reinforcement learning, simulated annealing or cvolutionary-based) are required to reliably reach the target ground state while simultaneously reducing quantum gate counts and circuit depth.
3.3 Tensor-based reinforcement learning
In contrast conventional methods, where we start from an empty RL-state 35 21 20
TensorRI-QAS starts from the warm-started circuit obtained using the procedures described above. The RL-agget receives such a circuit using the binary encoding scheme introduced in 35 <t> The agent then sequentially adds gates to this initialized structure until a stopping criterion is met, for example, if the maximum number of actions for that episode reached, or if target accuracy is achieved (typically, this is defined as the chemical accuracy for quantum chemistry problems). At each time step, the agent receives a modified RL-state representing the circuit with the newly proposed action. Each action is sampled from an action space containing a pool of gates, in our case set to (RX, RY, RZ, C To steer the agent towards the target, we use the same reward function R used in [1 at every time step of an episode (see Appendix M. for more details). With this framework mind, we introduce two different models, graphically represented in Fig. 2
TensorRL. (trainable TN-init 0 In the first method, we encode hoth the structure and the parameters of the warm-start circuit in the RL-state through the binary encoding, as illustrated in Fig. [a). This enables the agent to receive complete information regarding the MPS, and trains its parameters along with those of the newly added gates in the upcoming steps. In this setting, the size of the encoding is (DMPs + D) x N x (N + N1-quua,, where the first DMPS x N x (N + Niiqubit) encodes the MPS circuit structure of depth DMPs. For each depth, the 2-qubit gates are encoded in the first (N x N) elements, and the l-qubit gates are encoded into the remaining (Nr Ni-qubi X N) elements.
RensorRL (fixed TN-init) In the second method, illustrated in Fig. (b)) we do not feed the warm-start circuit to the RL-state, but rather use it as a fixed initial statevector for the following computations. This means that the binary encoding of the current RL-state does not contain information about the MPS, which is used as a fixed warm-start from the agent. Consequently, this reduces the size of the binary encoding tensor from (DMes + D) X N x (N + NNi-quiit to D) N (N + NN-quuit). In section Sec. 4 we show that this setting significantly reduces the computational time of each episode, and this happens for the following reasons: (i) due the reduced size of the input tensor and number of steps, the quantum statevector simulation time reduces (see Appendix D (if) as the RL-state size
'Trroughout the manuscript, the TensorRL (trainable TN-init) (TensorRL (fixed TN-init)) and TensorRL (trainable) (TensorRL (fixed)) are used interchangeably to denote the same method.
5

---

Figure 2: (a) In TensorRL (trainable TN-init), an MPS is transformed into brickwork circuit structure using Riemannian optimization. 'he circuit is encoded into a binary encoding to make visible to the RL-agent, The RL-agent chooses gate, and the information corresponding the gate is encoded into a new binary matrix, highlighted in red. In StructureRL, we follow the same steps as above but replace the parameters with zero. (b) TensorRL (fixed TN-init) do not directly encode the MPS structure and its parameters into the Rt-state, but the empty PQC is initialized with the MPS wavefunction.
RL
MPS
(a) TensorRI. (trainable TN-init)
RL
V MPS)x
|MMPsix
RL
(b) TensorRL (fixed TN-init)
reduces the neural network receives a reduced input, which further improves the training time per step: and (iii) since the warm-start circuit is non-traine non-trainable. the number of trainable parameters in the PQC is smaller, which in turn reduces the number of energy evaluations and the time taken by the classical optimizer. A brief discussion of the implementation of TensorRL (fixed TN-init) in quantum hardware is provided Appendix B
We additionally compare TensorRL techniques to StructureRL, a baseline where the RL-agent is initialized with the warm-start circuit structure (parameters set to zero) and can add gates or modify all parameters, including those in the warm-start section; this isolates the effect of warm- starting versus fixed ansatz initialization. Both TensorRL (fixed TN-init) and TensorRL (Irainable TN-init) yield the most compact PQCs among QAS baselines. With sufficient training, StructureRL, matches the performance of TensorRL (trainable TN-init), as both share the same initial structure and ultimately optimize similar parameters. Notably, TensorRL (fixe TN-init) accelerates episode execution, enabling efficient CPU-only training up to 8-qubit, reducing optimizer calls, and scaling RL-based QAS beyond 12-qubit, including noisy 8-qubit simulations.
Results
We begin this section by describing the variational quantum optimization problems used to evaluate TensorRI-QAS. We then detail the agent-enviromment specifications and present the results.
Experimental details We assess TensorRL-QAS on set of molecular quantum chemistry problems ranging from 6- 12-qubit, specifically targeting the ground state preparation of 6-BEH, 10-Hz0, and 12-LiH molecules from [Л]] (see Appendix M.3)). We benchmark against standard
[ 8-H₂O,
RL-QAS baselines, including CRLQAS 33], vanilla RL-QAS 21 371. RA-QAS B7], training-free
Mooeecless and their corresponding qubit requirements are labelled in the format "N pholecule_namme where N indicates the number of qubits.

---

ErpendiixEproviddee detailed descriptions of these methods. [ results for two CRLQAS variants, with CRLQAS (rerun) retrained under the same agent- envirommet settings (discussed below) for fair comparison. Beyond quantum chemistry, Appendix shows that TensorRI.-QAS also outperforms baseline QAS on the 5-qubit Heisenberg and 6-qubit TFIM models, yielding more compact circuits and reduced errors.
QAS 42). SA-QAS [3]. and quantumDARTS 41
Agent-cnvironment specifications Our implementation employs the double deep-Q network (DDQN) algorithm [62] with n = 5-step trajectory roll-outs 63], discount factor 7 = 0.88, and 6- greedy exploration where e decays geometrically from to 0.05 (decay rate 0. 0.99995/step). The target network is updated every 500 steps, with unified hyperparameters across quantum systems: batch size 1000, replay memory size 20000, ADAM optimizer [60] with learning rate 7 = 3 x 10 and 5 5-layer neural networks of 1000 neurons each. All Tensor -AS configurations use bond dimension x= unless otherwise specified, ensuring consistent entanglement structure throughout the search. For DMRG and Ces-tto-crrcuit mapping, we use the tensor network package by quimb [64]. The resulting MPS is then mapped to one-layered brickwork quantum circuit (for example, the first three unitaries in Eq. (3) using the process explained in Sec. 3.2 and then transpile to (RX, RY, RZ, CX). The maximum number of steps (i.e., gates) per episode detailed in Appendix M.2] which demonstrates that TensorRI--QAS needs fewer than half the steps per episode compared with baseline RL-based QAS. Moreover, in Appendix. D we elaborately discuss why the DDQN algorithm is utilized instead of other on-policy and off-policy algorithms.
4.1 Noiseless simulation
As shown in Tab. TensorRL (fixed TN-init) consistently produces more compact circuits while maintaining competitive or better accuracy across all molecular systems. For 6-BEH,, TensorRL (fixed TN-init) achieves 6.8 x 10 error with only 7 depths, CNOT's and 12 rotations-an order of magnitude improvement over RA-QAS and SA-QAS with 5-9x fewer CNOT'. For 8-H,0, it requires merely 6 depth and CNOTs versus 40-100 depth and 19-117 CNOTs for competing methods, while maintaining 8.9 x 10-4 error-a 7-13% x efficiency improvement. For 10-H20, TensorRL (fixed TN-init) achieves 4.1 x 10-4 error with just 15 CNOTs. While vanilla RL has slightly lower error, it requires 6x more CNOTs and 4x greater depths. For 12-LiH, TensorRL (trainable TN-init, X = 2) achieves the best error using only 37 CNOTs-a 10.5 x reduction versus SA-QAS and 8.7x versus vanilla RL. The fixed TN-init variant produces an exceptionally compact circuit with 15 depths, 30 CNOTs, and only 9 rotations. Moreover, TensorRL demonstrates perfect consistency, achieving chemical accuracy across 100% of seeds, while vanilla RL succeeds in only 70% of 10-H₂0 trials, RÁ-QAS in 60%, and SA-QAS in none. TensorRL's efficiency advantage scales with system size, growing from 5-9x CNOT reductions for 6-qubit systems to 10-13x x for 12-qubit systems. Importantly, we note that the numbers reported in the Table for TensorRL (fixed TN-init), only counts the operations added by the agent, which are the only ones it can see, since the warm-start is kept as statevector and not fed to the RLastate, (see See 3.3 for detailed discussion).
To further access the advantage of TensorRL (fixed TN-init), we optimize circuits using hardware- efficient ansatz (HEA) after the same MPS-to-cireuit mapping as fixed initialization, namely, TN-VQE. During the training each parameterized gate is optimized by COBYLA (with 1000 iterations). The results are presented in Tab. [ While TN-VQE VQE produces some improvement, it consistently yields higher errors and less efficient circuits than all RL-based approaches, and often fails to reach chemical accuracy, especially for larger molecules. This demonstrates that even with a high-quality initialization, RL-based circuit construction is crucial for achieving both optimal accuracy and efficiency.
In addition to accuracy and circuit compactness, in Fig. 3 we highlight the distinctive feature of TensorRL, namely its substantial computational efficiency. The left panel shows that TensorRL (fixed) consistently requires only ~ 10t function evaluations per episode, up to 100x fewer than competing methods, which instead require 100-10$ evaluations. While TensorRL (trainable) initially needs more evaluations, it rapidly converges to similar efficiency within 10 episodes. This reduction in function evaluations translates directly to lower computational overhead, as PQC simulation dominates resource usage. The right panel further underscores TensorRL's practical gains: TensorRL (trainable) ) and TensorRL (fixed) reduce per-episode execution time by 80% and 98%, respectively, compared to CRIQAS. These improvements stem from TensorRL's physically motivated initialization and exploration, which avoids expensive evaluation of suboptimal circuits. As a result, TensorRI. enables more thorough exploration and optimization within the same computational budget, making
7

---

Table TensorRL finds more compact circuit compared CRLQAS [35,, vanilla RL- QAS [21 371, random agent (RA) QAS (37), training-free QAS [42), simulated annealing (SA) QAS (39), quantumDARTS 11I, and TN-VQE (which consists fixed tensor network initialization followed by standard hardware efficient ansatz) approaches for 6-BEH2, 8-H20, 10-H20 and 12-LiH molecules. For TF-QAS 42 the total number of gates includes parameterized XX, YY and ZZ gates, which can further be decomposed into {RX, RY, RZ, CX). NA denotes not applicable, implying that the specific data for certain variables are not available.
Molecule
Method
Error
Depth
CNOT
ROT
6-BER²
TensorRL. (fixed TN-init)
6.8 10 10-6
7
12
TensorRL (trainable TN-init)
6.0 x 10
33
22
96
StructureRL
5.9 10 10-5
33
21
97
TF-QAS
1.8 x 10
NA
57 (total gate)
RA-QAS
37
5.8 x 10
21
26
17
SA-QAS
5.6 x 10
36
TN-VQE
5.4 x 10
10
8
12
8-H₂0
TensorRL (fixed TN-init)
8.9 x 10
6
9
15
TensorRL (trainable TN-init)
2.0 10
36
146
StructureRL
1.3 x 10
33
30
133
CRLQAS
35
1.8 10
75
105
35
CRLQAS (rerun)
1.7 x 10
100
85
58
Vanilla RL 21
x 10
96
117
48
quantumDARTS
x 10
64
68
151
RA-QAS
x
56
87
41
SA-QAS
2.6
40
26
TN-VQE
2.7 10
10
14
6
10-HęO
TensorRL (fixed TN-init)
4.1 10
17
15
17
TensorRL. (trainable TN-init, X 3)
x.7
33
168
TensorRL. (trainable TN-init) StructureRL
7.1
35
48
173
5,8 10
37
169
CRLQAS (rerun)
3 10
114
140
43
Vanilla RL
x 10
73
96
32
RA-QAS
9.9 x 10
80
152
58
SA-QAS
x 10
24
42
14
TN-VQE
1.1 x 10
26
14
12-LiH
TensorRL (trainable TN-init)
1.0 x 10 10
31
37
203
TensorRL (fixed TN-init)
2.4 10
15
30
9
StructureRL
2.2 x 10
40
179
CRLQAS (rerun)
x
32
68
23
Vanilla RL
x 10
140
94
RA-QAS
2.3 x 10
165
364
86
SA-QAS
2.5 10
178
TN-VQE
8.6 x 10
33
81
29
especially valuable for resource-intensive molecular simulations. Overall, the 10-100x reduction in function evaluations and multiple titer-of-maanitude speedup establish TensorRI. as both more accurate and markedly more efficient than existing quantum architecture search (QAS) methods.
we show that TensorRL consistently enhances RL-agent trainability, yielding higher accumulated rewards than baselines. As problem size increases, in Appendix H.. TensorRL maintains a success probability of up to 50%, while other methods rarely exceed 1%. Additional experiments on 8- and 10-qubit CH₃O are reported in Appendix. H which confirm the advantage of TensorRL-QQAS in gate count, circuit depth, and accuracy. Fig. 6 illustrats that TensorRL-OAS converges to chemical accuracy up to 8-qubit on CPU with better training time than CRLQAS and vanilla RI, enabling practical QAS on a CPU. Finally, in Appendix. H.6) we show that with TensorRI. (fixed) we can find good approximation to the ground energy of the transverse field Ising model up 20-qubit. These results demonstrate both the versatility and strong performance of
In Appendix H.2

---

Figure 3: TensorRL (trainable) and TensorRL (fixed) require 80% and 98% less time, respec- tively, to execute an episode compared to CRLQAS algorithms. Meanwhile, TensorRL (trainable) and TensorRL (fixed) improve the number of function evaluations (nfev) by the classical optimizer by 10-100 fold, availing the corridor for more fixed computational budget. The error bar (on the right figure) is the standard deviation (a) from the average over 5 random initializations of the neural network. The molecule in this example is 8-qubit H2O.
pelooation-exploitatioo by the e-greedy agent in a
nfev per episodes
Time of execution
10
Teree-Rl RI (tninable x 2)
per spisade
níev in classical optimizer
10
(fxed)
spisode
StructurRL
Varilla RL
Time (in seconds)
CRLQAS (rerun)
100
10
10a
10
10
TimeRR (feel)
Stecturfft
CRLOAS (reun)
Episodes
Tewarft (miinable x
Vanila
our approach across diverse quantum systems. We further analyze the agent's refinement on tensor network initialization as function of DMRG bond dimension, with results provided in Appendix Table This scaling analysis demonstrates that TensorRL offers diminishing improvements as DMRG approaches chemical accuracy, highlighting that the principal advantage of TensorRL-QAS emerges in regimes where classical tensor methods become intractable.
4.2
Noisy simulation
In the previous section, we conducted simulations in the ideal noiseless scenario, and now we report results for running TensorRL-QAS in a noisy setting. Specifically, we consider the 8-qubit H₂O molecule ground-state search problem using depolarizing noise rates exceeding current IBM quantum (65) hardware: single-qubit gate errors were amplified 10x (i.e. 1%). 2-qubit errors 5x (i.c. 5%), and shot-noise from 10a samples was included (for details see Appendix F). While these conditions do not fully capture all device-specific noise characteristics, they provide challenging and controlled benchmark for QAS methods. Notably, to the best of our knowledge, this is the largest simulations to date of RL-QAS
noisy setting, which were previously limited to at most 4-qubit (38 335. Under these conditions, TensorRL achieved up to 38% improvement in the error for ground state preparation and 2.4x reduction in average circuit depth compared to the baseline noisy QAS method, CRLQAS (35, indicating improved resilience in both realistic and challenging noise environments. To demonstrate TensorRL-QAS" robustness to hardware noise, we take PQCs proposed during noiseless training that reach the ground state and execute them on IBMQ-Bri her--Brisband device. Results Appendix Conffir that TensorRL-QAS can achieve chemical accuracy under realistic hardware noise.
As shown in Table 2 TensorRL (fixed) achieved chemical accuracy with 100% success rate under depolarizing noise, compared to CRLQAS' 30% success rate at comparable error. This performance difference is linked TensorRL's strategy of preserving noiseless reference states during policy updates, potentially insulating the learning process from noise corruption. The method also produced shallower circuits with half of the CNOT gates compared to CRLQAS, which may help mitígate error accumulation under the amplified 2-qubit noise conditions. Under shot noise, TensorRL. maintain an error below chemical accuracy with only a depth circuit, though we note this simplified noise model does not account for correlated errors crosstalk effects present in physical devices.

---

Table 2: TensorRL outperforms baseline CRLQAS under 1- and 2-qubit depolarizing. We tackle the problem of finding the ground state of 8-H2O.
Noise
Method
Error
Depth
CNOT
ROT
Success prob.
Depolarizing
TensorRL (fixed)
9.0 x 10
7
12
100%
TensorRL (trainable)
2.48 x 10-3
33
22
130
0%
StructureRL
2.6 10-3
28
129
0%
CRLQAS (rerun)
1.3 10
22
11
29
30%
Shot
TensorRL (fixed)
1.5 10
3
4
100%
TensorRL (trainable)
8.7 10
30
131
100%
StructureRL
6.8 10
34
25
135
100%
Conclusion and discussion
We introduced TensorRL-QAS, an improved framework for quantum architecture search (QAS) that combines tensor network initialization with reinforcement learning. Leveraging a matrix product state initialization, TensorRL-QA reduces function evaluations by x00 and achieves 98% faster per-episode execution compared to baseline QAS methods [42 accuracy. For 6- to- to-12-qubit chemical Hamiltonians, it consistently yields circuits with 10-13x fewer CNOTs and depth, robust performance across random seeds, better efficiency and stability relative to RI. and non-RI. QAS baselines. The approach generalizes to non-chemical tasks, supports CPU-only training up to 8-qubit, and maintains high accuracy under amplified noise, achieving up to 38% lower energy error and 2.4x smaller circuit depth than existing noisy-QAS methods. These results position TensorRL-QAS as a robust, efficient, and noise-adaptive solution for quantum circuit discovery on near-term devices and in the fault-tolerant quantum computing era [67
35 18 66 39 371, without sacrificing
Limitations and future directions We study two TensorRL-QAS models: where the warm- start is part the RL state (and thus agent-Irainable), and one where it is not. Exploring intermediate variants-e.g, where the agent observes but cannot modify the warm-start circuit- remains open. Our action space was limited to single-qubit RX, RY, RZ rotations and CNDT gates; as qubit count and problem complexity grow, richer action spaces may be required. Incorporating proecic- (e.g., Hamiltonian -inspired) gates [16 68 69 or exploiting symmetries and repeated structures (e.g., gadgets 770 (1]) could improve performance beyond 12-qubit systems. Preliminary results with a modified action space presented in Appendix. H.6). moreover, here we also point out the scaling limits of the action space. Further, we only a brickwork warm-start circuit, matching typical QPU connectivity and MPS structure; future work could examine alternative ansatz and tensor network layouts 722 59). Ultimately, due to long queue times on real QPUs, our experiments rely on noise simulations, where we artificially amplify noise by 5-10x to test the robustness of TensorRL-QAS However, have not conducted training on actual quantum hardware. Future work could focus on real-device training and on developing methods to adapt TensorRL-QAS to actual QPU noise.
In summary, the methods proposed in this work provide ofate-of- the-art results in quantum architecture search and establish a resource-efficient framework towards scalable reinforcement learning- variational quantum optimization. TensorRL-QAS paves the way for practical RL-based quantum circuit design across diverse variational quantum algorithms and larger system sizes. We believe numerous follow-up avenues will reveal the true capability of TensorRL-QAS.
Code The source code for all experiments in this manuscript is accessible here:
6 Acknowledgment
The authors acknowledge CSC IT Center for Science, Finland, for computational resources. AK and SM acknowledge funding from the Research Council of Finland through the Finnish Quantum Flagship Project 358878 (UH). The authors are grateful to the anonymous NeurlPS reviewers and ACs for valuable comments and insights that helped us improve the research.
References
[1] John Preskill. Quantum computing in the nisg and beyond. Quantum, 2:79, 2018.
10

---

[2] Thomas Monz, Daniel Nigg, Esteban A Martinez, Matthias F Brandl, Philipp Schindler, Richard Rines, Shannon X Wang, Isaac L Chuang, and Rainer Blatt. Realization scalable shor algorithm. Science, 351 1068777)))66810777776 2016. T
[3] Alberto Peruzzo, Jarrod McClean, Peter Shadbolt, Man-Hong Yung. Xiao-QU Qi Zhou, Peter J Love, Alán Aspuru Aspuru-Guzik, and Jeremy L O'brie. variational eigenvalue solver on a photonic quantum processor. Nature communiccations, 5(1))4213, 2014.
[4] Jarrod R McClean, Jonathan Romero, Ryan Babbush, and Alán Aspuru-Guzik. The theory of variational hybrid quantum-classical algorithms. New J. Phys., 18(2):22023, 2016. |
[5] Marco Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, et al. Variational quantum algorithms. Nature Reviews Physies, 3(9):625-644, 2021. 2
[6] Kishor Bharti, Alba Cervera-Lierta, Thi Ha Kyaw, Tobias Haug. Sumner Alperin-Lea, Abhinav Anand, Matthias Degroote, Hermanni Heimonen, Jakob S Kottmann, Tim Menke, et al. Noisy intermediate-scale quantum algorithms. Reviews of Modern Physics, 94(1):015004, 2022.
[7] Abhinav Kandala, Antonio Mezzacapo, Kristan Temme, Maika Takita, Markus Brink, Jerry M Chow, and Jay M Gambetta. Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets. nature, 549(767 1):242-22222 226,, 2017.
[8] Andrea Skolik, Michele Cattelan, Sheir Yarkoni, Thomas Bäck, and Vedran Dunjko. Equivariant quantum circuits for learning on weighted graphs. npj Quantum Information, 9(1) May 2023,
[9] Johannes Jakob Meyer, Marian Mularski, Elies Gil-Fuster, Antonio Anna Mele, Francesco Arzani, Alissa Wilms, and Jens Eisert. Exploiting symmetry in variational quantum machine learning. PRX Quantum, 4(1), March 2023. 2
[10] Martín Larocca, Frédéric Sauvage, Faris M. Sbahi, Guillaume Verdon, Patrick J. Coles, and M. Cerezo. Group-invariant quantum machine learning. PRX Quantum, 3(3), September 2022.
[11] Martín Larocca, Supanut Thanasilp. Samson Wang, Kunal Sharma, Jacob Biamonte, Patrick J. Coles, Lukasz Cincio, Jarrod R. McClean, Zoë Holmes, and M. Cerezo. Barren plateaus in variational quantum computing. Nature Reviews Physics, 7(4):174-119, March 2025. 2
[12] Manuel Rudolph, Jacob Miller, Danial Motlagh, Jing Chen, Atithi Acharya, and Alejandro Perdomo-Ortiz. Synergistic pretraining of parametrized quantum circuits via tensor networks. Nature Communications, 14(1), December 2023.
[13] Steven White. Density matrix formulation for quantum renormalization groups. Physical Review Letters, (2119)):88332286, November 1992. 2
[14] Ulrich Schollwöck. The density-matrix renormalization group in the age of matrix product states. Annals of Physics, 326(1):96-19,, January 2011.
[15] Ilia A Luchnikov, Mikhail E Krechetov, and Sergey N Filippov. Riemannian geometry and auto- matic differentiation for optimization problems of quantum physics and quantum technologies. New Journal of Physics, 23(7))073006,, July 2021. 2 3
19
20
[16] Harper R Grimsley, Sophia E Economou, Edwin Barnes, and Nicholas J Mayhall. An adaptive variational algorithm for exact molecular simulations on a quantum computer. Nature comm., 10(1):1-9, 2019. 3 10
25
26
29
[17] Ho Lun Tang, VO Shkolnikov, George S Barron, Harper R Grimsley, Nicholas J Mayhall, Edwin Barnes, and Sophia E Economou. hardware-efficien ansätze quantum processor. PRX Quantum, 2(2):020310, 2021.
An adaptive algorithm for constructing
[18] Shi-Xin Zhang, Chang-Yu Yu Hsieh, Shengyu Zhang, and Hong Yao. Differentiable quantum architecture search. Quantum Science and Technology, 7(4):045023, 2022.
[19] En-Jui Kuo, Yao-Lung I. Fang, and Samuel Yen-Chi Chen. Quantum architecture search via deep reinforeement learning, arXiv preprint arXiv:2104.07715, 2021. 2
5

---

[20] Thomas Fösel, Murphy Yuezhen Niu, Florian Marquardt, and Li Li. Quantum circuit opti- mization with deep reinforcement learning. arXiv preprint arXiv:2103.07585, 2021.
5
[21] Mateusz Ostaszewski. Lea M Trenkwalder, Wojeiech Masarczyk, Eleanor Scerri, and Vedran Dunjko. Reinforcement learning for optimization of variational quantum circuit architectures. Advances in neural information processing systems, 1118888--8194 2021.
5
6
S
26
32
[22] Owen Lockwood. Optimizing quantum variational eircuits with deep reinforcement learning. arXiv preprini 1.N.:2209900//88 2021. 2
[23] James Dborin, Fergus Barratt, Vinul Wimalaweera, Lewis Wright, and Andrew G Green. Matrix product state pre-training for quantum machine learning. Quantum Science and Technology, 7(3):035014, may 2022.
[24] Aleksandr Berezutskii, Atithi Acharya, Roman Ellerbrock, Johnnie Gray, Reza Haghshenas, Zichang He, Abid Khan, Viacheslav Kuzmin, Minzhao Liu, Dmitry Lyakh, Danylo Lykov, Salvatore Mandrå, Christopher Mansell, Alexey Melnikov, Artem Melnikov, Vladimir Mironov, Dmitry Morozov, Florian Neukart, Alberto Nocera, Michael A. Perlin, Michael Perelshtein, Ruslan Shaydulin, Benjamin Villalonga, Markus Pflitsch, Marco Pistoia, Valerii Vinokur, and Yuri Alexeev. Tensor networks for quantum computing, 2025. 2
[25]
Hans-Martin Rieser, Frank Köster, and Amn Peter Raulf. Tensor networks for quantum machine learning. Proceedings of the Royal Society A, 479(2275):20230218 2023. _
[26] Tomislav Begušić, Johnnie Gray, and Garnet Kin-Lic Chan. Fast and converged classical simulations of evidence for the utility of quantum computing before fault tolerance. Science Advances, 10(3), January 2024.
[27] Joseph Tindall, Matthew Fishman, E. Miles Stoudenmire, and Dries Sels. Efficient tensor network simulation of ibm's ' eagle kicked ising experiment. PRX Quantum, 5(1), January 2024.
[28] Stefano Mangini, Marco Cattaneo, Daniel Cavalcanti, Sergei Filippov, Mattco A. C. Rossi, and Guillermo Garefa-Pérez Tensor network noise characterization for near-term quantum computers. Physical Review Research, 6(3), August 2024. 2
[29] Giacomo Torlai, Christopher J. Wood, Atith Acharya, Giuseppe Carleo, Juan Carrasquilla, and Leandro Aolita. Quantum process tomography with unsupervised learning and tensor networks. Nature Communications, 14(1), May 2023. 20
[30] Yuchen Guo and Shuo Yang. Quantum state tomography with locally purified density operators and local measurements. Communications Physics, 7(1), October 2024. 2
[31] Sergei Filippov, Matea Leahy, Matteo A. C. Rossi, and Guillerno Garcia-Péres Scalable tensor-network error mitigation for near-term quantum computing, 2023.
2 21
[32] Laurin E. Fischer. Matea Leahy, Andrew Eddins, Nathan Keenan, Davide Ferracin, Matteo
A. C. Rossi, Youngseok Kim, Andre He, Francesca Pietracaprina, Boris Sokolov, Shane Dooley, Zoltán Zimborás, Francesco Tacchino, Sabrina Maniscalco, John Goold, Guillermo García- Pérez, Ivano Tavernelli, Abhinav Kandala, and Sergey N. Filippov. Dynamical simulations of
many-body quantum chaos on a quantum computer, 2024. 2
[33] En-Jui Kuo, Yao-Lung I. Fang, and Samuel Yen-Chi Chen. Quantum architecture search via deep reinforcement learning. arXiv preprint 0rXiv:2 2021. 3
[34] Esther Ye and Samuel Yen-Chi Chen. Quantum architecture search via continual reinforcement learning. arXiv preprint 1 2021.
[35] Yash Patel, Akash Kundu, Mateusz Ostaszewski, Xavier Bonet-Monrois Vedran Dunjko, and Onur Danaci. Curriculum reinforcement learning for quantum architecture search under hardware errors. In The Twelfth International Conference on Learning Representations, 2024.
3
5 6
g
g
25
26 32
12

---

[36] Francisco JR Ruiz, Tuomas Laakkonen, Johannes Bausch, Matej Balog, Mohammadamin Barekatain, Francisco JH Heras, Alexander Novikow, Nathan Fitzpatrick. Bernardino Romera- Paredes, John van de Wetering, et al. Quantum circuit optimization with alphatensor. Nature Machine Intelligence, pages 1-12, 2025.
[37] Akash Kundu, Przemystaw Bedelek, Mateusz Ostaszewski, Onur Danaci, Yash J Patel, Vedran Dunjko, and Jarostaw A Miszezak, Enhancing variational quantum state diagonalization using reinforcement learning techniques. New Journal of Physics, 26(1):013034, 2024.
3
S 10
23
[38] Yuxuan Du, Tao Huang, Shan You, Min-Hsiu Hsich, and Dacheng Tao. Quantum circuit architecture search for variational quantum algorithms. upj Quantum Information, 8(1):62,
2022.
[39] Xudong Lu, Kaisen Pan, Ge Yan, Jiaming Shan, Wenjie Wu, and Junchi Yan. Qas-bench: rethinking quantum architecture search and benchmark. In International conference machine learning. pages 22880-22898 22898. PMLR, 2023.
g
10
33
[40] Li Ding and Lee Spector. Evolutionary quantum architecture search for parametrized quantum circuits. In Proceedings of the genetic and evolutionary computation conference companion, pages 2190-2101 2195, 2022.
[41] Wenjie Wu, Ge Yan, Xudong Lu, Kaisen Pan, and Junchi Yan. Quantumdarts: Differentiable quantum architecture search for variational quantum algorithms. International Conference Machine Learning., 40, 2023.
[42] Zhimin He, Maijie Deng, Shenggen Zheng, Lvzhou Li, and Haozhen Situ. Training- free quantum architecture search. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 12430-12438 2024.
TO 22 29
[43] Hanrui Wang, Yongshan Ding, Jiaqi Gu, Yujun Lin, David Z Pan, Frederic T Chong, and Song Han. Quantumnas: Noise-adaptive search for robust quantum circuits. In 2022 IEEE International Symposium on Peigh-Perfrmancce Computer Architecture (HPCA), pages 692-708.
IEEE. 3
[44] Aqasch. awesome-qas. https: https://gttub..com///eo// http:tuub.cmmmggaccca//eesoooooo QAS 2025. Accessed:
2025-09-24.8 3
[45] Darya Martyniuk, Johannes Jung, and Adrian Paschke. Quantum architecture search: survey. In 2024 IEEE International Conference on Quantum Computing and Engineering (QCE). volume 1, pages 1695-1706. IEEE, 2024. 3
[46] Akash Kundu. Reinforcement learning- assisted quantum architecture search for variational quantum algorithms. arXiv preprint 2402. 13754, 2024.
[47] Hemant D. Tagare. Notes on optimization on stiefel manifolds, 2011. 19
[48] Nicolas Boumal. An introduction to optimization on smooth manifolds. Cambridge University Press, 2023. 4 19
[49] Román Orús. A practical introduction to tensor networks: Matrix product states and projected entangled pair states. Annals of Physics, 349:117-158, October 2014.
[50] G. Catarina and Bruno Murta. Density renormalization group: pedagogical introduction. The European Physical Journal B, 96(8), August 2023.
[51] Alberto Baiardi and Markus Reiher. The density matrix renormalization group in chemistry and molecular physics: Recent developments and new challenges. The Journal of Chemical Physics, 152(4), January 2020. 4
[52] V. Oseledets. Tensor-train decomposition.
SIAM Journal on Scientific Computing,
33(5):2225-23111 January 2011.
13

---

[53] Sergey V. Dolgov and Dmitry V. Savostyanov. Alternating minimal energy methods for linear systems in higher dimensions. SIAM Journal on Scientific Computing, 3665):A2448-A2277 January 2014.
[54] C. Schön, E. Solano, F. Verstraete, J. I. Cirac, and M. M. Wolf. Sequential generation of entangled multiqubit states. Physical Review Letters, 95(11), September 2005.
[55] Shi-Ju Ran. Encoding of matrix product states into quantum circuits of one- and two-qubit gates. Phys. Rev. A, 101:032310, Mar 2020.
[56] Matan Ben-Dov, David Shnaiderov, Adi Makmal, and Emanuele Dalla Torre Approximate encoding of quantum states using shallow circuits. npj Quantum Information, 10(1), July 2024.
20
[57] Manuel S Rudolph, Jing Chen, Jacob Miller, Atithi Acharya, and Alejandro Perdomo-Ortiz Decomposition of matrix product states into shallow quantum circuits. Quantum Science and Technology, 91):01501,, nov 2023,
19
[58] Ar A Melnikov, A A Termanova, S V Dolgov, F Neukart, and M R Perelshtein. Quantum state preparation using tensor networks. Quantum Science and Technology, 8(3):035027, 2023.
[59] Reza Haghshenas, Johnnie Gray, Andrew C. Potter, and Garnet Kin-Lic Chan. Variational power of quantum circuit tensor networks. Physical Review X, 12(1), March 2022.
[60] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint
arXivv1412.6980, 2014. 5 20
[61] Pennylane. Top 20 molecules for quantum computing.
httpos//peum/Laae..a/bbeeg
2024/01/top- 3000mole www.ees-ooo---eeeeeeeeeeoooooeee for
2025. Accessed: 2025-04-25. 33
[62] Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q q-learning. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
22
[63] Richard Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press,
2018. 21
[64] Johnnie Gray. quimb: A python package for quantum information and many-body calculations.
Journal of Open Source Software, 3129):819, September 2018. 7
[65] IBM Corporation. IBM Quantum. https://quattuu-cooputiiii ibm. wwt.e/oo..s..eeeeee services/ resources 2025, Accessed: 2025-04-28, 24
[66]
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprini arXiv: 180.00050 2018.
TO
[67] Zoltán Zimborás, Bálint Koczor, Zoë Holmes, Elsi-Mari Borrelli, András Gilyén, Hsin- Yuan Huang, Zhenyu Cai, Antonio Acín, Leandro Aolita, Leonardo Banchi, et al. Myths around quantum computation before full fault tolerance: What no-go theorems rule out and what they
don't. arXiv preprini arXiv:2501 1556944 2025. 10
[68] Adam Glos and Özlem Salehi. A generic multi- compilation framework for limited connectivity. arXiv preprint arXiv:2412.0660 069099 2024. I0
[69] Ioana Moflic and Alexandru Paler. On the constant depth implementation of pauli exponentials. arXiv preprint wxXiv:2408.08266
2024. TO
[70] Akash Kundu and Leopoldo Sarra. Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware. arXiv preprint arXiv:2411.002330 2024. I0
[71] Jan Ollé Aguilera, Oleg M Yevtushenko, and Florian Marquardt. Scaling the automated discovery of quantum cireuits via reinforcement learning with gadgets. arXiv preprint 11638,, 2025. I0
14

---

[72] Matias Bilkis, Marco Cerezo, Guillaume Verdon, Patrick J Coles, and Lukasz Cincio. A semi- agnostic ansatz with variable structure for variational quantum algorithms. Quantum Machine Intelligence, 5(2):43, 2023. TO
[73] Isabel Nha Minh Le, Shuo Sun, and Christian B. Mendl. Riemannian quantum circuit optimiza- tion based on matrix product operators, 2025. 19 20
[74] Alan Edelman, Tomás A. Arias, and Steven T. Smith. The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications, 20(2):303-353,
1998. 19
[75] Ilia A. Luchnikov, Alexander Ryzhov, Sergey N. Filippov, and Henni Ouerdane. QGOpt: Riemannian optimization for quantum technologies. SciPost Phys., 10:079, 2021. 20
[76] Azhar Ikhtiarudin, Aditi Das, Param Thakkar, and Akash Kundu. BenchRI-QQSS: Bench- marking reinforcement learning algorithms for quantum architecture search. arXiv preprint arXiv:2507.12 1219, 2025. 21
[77] Yuhui Wang, Hao He, and Xiaoyang Tan. Truly proximal policy optimization. In Uncertainty in artificial intelligence, pages 113-122. PMLR, 2020. 21
[78] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint
06347, 2017. 21
[79] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforce- ment learning. In International conference on machine learning, pages 1928-19299 1937. PmLR,
2016. 21
[80] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. 2T
[81] Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. Dueling network architectures for deep reinforcement learning. In International conference on machine learning, pages 1995-2003. PMLR, 2016.
21
[82] Andrew M Childs, Eddie Schoute, and Cem M Unsal. Circuit transformations for quantum architectures. arXiv preprint 10Xv011022.09102, 2019. 22
[83] Michael JD Powell. direct search optimization method that models the objective and constraint functions by linear interpolation. Springer, 1994. 23
[84] Miroslav Urbanek, Benjamin Nachman, Vincent R Pascuzzi, Andre He, Christian W Bauer, and Wibe A de Jong. Mitigating depolarizing noise on quantum computers with noise-estimation circuits. Physical review letters, 127(27)220002,, 2021.
24
[85] Carlo Beenakker and Christian Schönenberger. Quantum shot noise. Physics Today, 56(5):37-
42, 2003. 24
[86] Zhimin He, Jiachun Wei, Chuangtao Chen, Zhiming Huang, Haozhen Situ, and Lvzhou Li. Gradient-based optimization for quantum architecture search. Neural Networks, 179:106508, 2024. 29
[87] Anton Nykanen, Leander Thiessen, Elsi-Mari Borrelli, Vijay Krishna, Stefan Knecht, and Fabijan Pavosevic. Toward accurate calculation of excitation energies on quantum computers with dadapt-vge: case study of bodipy derivatives. The Journal of Physical Chemistry Letters,
15(28)7711-7117, 2024.
29
[88] Yasunari Suzuki. Yoshiaki Kawase, Yuya Masumura, Yuria Hiraga, Masahiro Nakadai, Jiabao Chen, Ken M Nakanishi, Kosuke Mitarai, Ryosuke Imai, Shiro Tamiya, al. Qulaes: a fast and versatile quantum circuit simulator for research purpose. Quantum, 5:559, 2021.
32 34
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our claim that TensorRL-QAS not only outperforms existing baseline QAS approaches in noisy and noiscless scenario but also is effectively scalable to larger quan- tum systems is clearly stated in the abstract and introduction. Along with the claim of computational advantage is also stated.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: A detailed discussion on the limitations of our work is provided in Sec.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: There are theoretical claims in the work. The paper is based on experiments and numerical results.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The steps to implement algorithm elaborately discussed in Section Moreover, the code to reproduce the results of the paper is currently available at URL. https///gghubb..om///aacccc///// Aww.aaaasssseeeooRL-AAAAA
Justification:
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: the code to reproduce the results of the paper is currently available at URL: https::/githu..com// Coomm/AAssc//eensrRR----o-
The README clearly states the environment requirements and commands to run the TensorRL--QAS algorithm.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The experimental settings are elaborated in the Appendix. ??, Ap-
pendix M.I Appendix [2 and Appendix 13 https://gitubb.oo///pgoo TensorRL-QAS
The code openly available URL:
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We clearly state that the error bars appearing in Fig. and Fig. represents the --iigm,, the maxima and minima values respectively, and the shaded area across the average in Fig. 4is is I-sigma over 5 random initialization of the neural network.
16

---

8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Tab. N we provide the memory, time of execution and the GPU and CPU utilized.
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpa: http:///www..ancc//////////iiiaa
Answer: [Yes]
Justification: Yes, both authors read the NeurIPS Code of Ethics and we confirm that the current paper conforms, in every respect, with the NeurlPS Code of Ethics.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
In Appendix. O we elaborate the borderline impact of our paper.
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses such risks.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the existing codes for baseline methods utilized in the paper and in the shared repository we provide links to the assets.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: In the code available in URL:
we include the details about training, license and limitations, and other important information.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects. 15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
17

---

Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
16. Declaration LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: Yes, LLMs are primarily utilized at the sentence level to fix grammar, reword sentences. In the Appendix, at the paragraph level for reorganizing sentences in a paragraph.
18

---

A Comparison to prior TN-PQC synergistic frameworks
While both our method and the approach of Rudolph et al. [2] harness tensor networks (TNs) to enhance parametrized quantum circuits (PQCs), there are several key differences between the two works:
Integration with reinforcement learning and quantum architecture search: Rudolph et al. [1] extend the quantum circuit obtained from an MPS warm-start with a fixed PQC structure initialized with random parameters, and then optimize all parameters the eircuit to further decrease the cost function. In contrast, TensorRL-QAr does not add fixed PQC but rather uses reinforcement learning (RL) within a quantum architecture search framework to dynamically explore and optimize the additional PQC architectures that best help in minimizing the loss. Such flexible exploration allows for balanced search of architectural expressivity and circuit compactness, and hence noise resilience, capability that is not present in fixed PQC schemes.
Trainability of PQCs vs. scalability of RL-QAS: The main focus of [12] is mitigate barren plateaus and improve trainability of PQCs through MPS-based initializations. On the contrary, our work focuses on alleviating the heavy computational requirements required by RL-based QAS procedures, thus making them practical also on larger system sizes. Indeed, we show that our approach can achieve 98% reduction classical computational overhead by warm-starting RL-framework with MPS warm-starts.
Different set of basis gates: Compared to 12], in this work, we use different and simpler set of basis gates, which is closer operations that are natively available on quantum hardware, and thus reduces potential compilation overheads.
Different MPS-to-PQC mapping scheme: In 12], the authors use a combination of explicit constructions and trainable operations to build the PQC approximating the MPS warm-start (57). In this work, instead, we use a fully variational method to find such a mapping based on Riemannian optimization [5 59 73).
B Mapping an MPS to quantum circuit via Riemannian optimization
In this appendix, we explain in more detail the procedure used to find quantum circuit representation of MPS using Riemannian optimization.
We consider the case where one fixes the structure of the quantum circuit, and aims to find the 2-qubit unitaries that maximize the overlap with the target quantum state. Specifically, let (V) denote the target MPS, and let U = II k= Us denote the unitary representing a quantum circuit with a given structure, where each of the UR is q-qub gate. We then want to solve the following optimization problem
M
M
{UE} = min L(FU))). (((UR)) (110)0)) = ( II (Ua)
U 0)
(4)
k=1
with Ug U(4)= (U EC -
[.} Vk 1l. M
that is. we want to perform optimization on the manifold of unitary matrices, the so-called com- plex Stiefel manifold (and we consider the simpler case unitary matrices rather than isometries). Such problem is solved using Riemannian optimization techniques aimed specifically at solving
optimization problems on manifolds 774 48 15 47).
We solve the minimization problem above by adapting gradient-descent methods so that each of the trainable parameters of the cost function L(-), namely the matrices {U+), remain unitary during the whole optimization process. Our implementation largely follows [15], and we refer the interested reader such a reference for an in-depth look Riemannian optimization, specifically for quantum information tasks. We note that since we are considering optimization over unitary matrices (rather than isometries), the Euclidean and canonical metric on Stiefel manifold coincide (74), which greatly simplifies the formulas, for example, compared to those reported in [5). Additionally, in order to move from one point to another on the manifold, we use Cayley's 's retruction, as it was seen to improve convergence compared to other methods, based for example on SVD [15.
19

---

Let (U......
denote the trainable unitaries at time step during optimization. Then, follow- ing 75,, an optimization procedure for adapting Adam [60] on the Stiefel manifold can be obtained as follows:
) .....U.
Ubr
1. Start optimization (t 0) from a point the manifold, that is, start with unitary matrices {UT,... UM }or with U E (4) Yk = l,..., ,...,,,.. M. For each of them, set the corresponding initial momentum and velocity terms
mi
t= 1=0 0 Vk 1.......A M;
2
where 0 is matrix of shape 4 x containing zeros;
2. For k = -,...,, M, compute the (Euclidean) gradients of the loss function at the current points
{
0L aUT
OL
aU
where each of the partial derivatives itself a 4 complex matrix. Importantly, since the variables are complex, the conjugate of the gradients is used in the following steps 0 + (aL/aUg))
3. For k = 1,, ................................ .M. compute the Riemannian gradients given the (Euclidean) gradients from the previous step (see Eq. (14) in (1S]
VRL(U OL
aL
U
U
aug
aU
4. For k .,.,, M, compute the updated momentum and velocity terms
m 333m4 + (1 -.) LU)
(8)
ot = 3202 + (1 - 32) Tr [ L(U VRL(U
where Tr TU)) L(L( = (VRL(U) L(U)) is the inner product of the Rieman- nian gradients given the metric. It is the equivalent of the gradient squared term in usual Adam (see also 73)..
5. For k L..... M,, compute the update direction as
0++1 mi / (Vot' +
where the square root and division are meant element-wise.
6. Apply bias-correction to the learning rate r + 7/1 B5/(1 (1)
7. For l 1,..., M, move to another point on the manifold using Cayley's retraction (see Eq. (15) in [15] specialized for unitary matrices)
UL+1 R
(-nūū)
(10)
RCayloy (V) =
(1-
1
W
11
W 2
U,
= VU UtV
2
2
8. For ls 1,..., M, use vector transport to the momentum and velocity terms the new points on the manifold (see Eq. (16) in (15)
1 with (V) = ! (v -v-vv. (11)
m ++1 Pogeo (m 1+1+)
of r+1
Pot (btt))
9. Repeat steps (2) to (8) until convergence.
We compute the Euclidean gradients in step (2) using automatic differentiation tensor network representation of the loss function in Eq. 4 Alternatively, the gradients with respect to each of the unitaries can be obtained by the standard "punching hole" method for computing gradients in tensor networks (29 73). Also, we remark that we perform -reddeentdeecent step on all unitaries in the variational circuit al the same time, that is, we simultaneously update them in a single step of the optimization loop. Other approaches are, however, possible. For example, a DMRG-like optimization procedure could optimize single unitary while keeping the remaining ones fixed, and then sweep over all unitaries until convergence [59 56).
20

---

C
Implementing TensorRL (fixed TN-init) on QPU
Recalling that in the TensorRL--AA (fixed TN-init) the training begins from a specially prepared tensor network state (such as an MPS). As detailed in Fig. 2b), rather than embedding information about the initial state in the RL-agent's observations, the MPS is treated non-trainable warm-start, significantly reducing the input tensor size and computational requirements.
To translate this idea to real quantum hardware, one can start by directly beginning with the fixed quantum circuit structure corresponding to MPS and then transpile this circuit to the target hardware, ensuring compatibility with device-specifi gate sets and connectivity. Any fidelity loss due to hardware noise can be addressed using error mitigation methods such as ref. [31] to better match the realized state to the idealized ini. With the TN-init successfully created on the device, the TensorRL. algorithm continues from this fixed base state: policies are learned and actions are taken atop this robust initialization, with error mitigation applied as necessary through the computation to maintain performance.
This workflow offers realistic and accessible route for deploying TensorRL with advanced initializa- tion on hardware. While necessitates overhead from error mitigation, the resulting process is both experimentally feasible and computationally efficient. Full-scale experimental demonstration is left as future work, given the current constraints in hardware access and resource availability.
D Why DDQN over other RL-algorithms?
In this section, we will elaborate on the motivation behind utilizing the DDQN algorithm. A direct comparison of different RL algorithms for QAS would be redundant, as the recent BenchRL-QAS [76]
study provides comprehensive benchmark of both policy-based and off-policy agents, including, TPPO 777. PPO 78. A2C (8) A3C (79), DQN (0), Dueling DQN 181, and DDQN 62]. In the
setting of ground state preparation with VQE for 6 BEH2, 8 H2O, and 10 H2O molecules, BenchRL- QAS is evaluated under simulated noise, showing that DDQN is the only agent that simultaneously achieves chemical accuracy and the lowest circuit depth. Although these benchmarks are performed in simulated environments, this balance between accuracy and circuit compactness directly translates into practical advantages on real noisy quantum hardware, where reducing gate count lowers error accumulation and increases fidelity. Motivated by these findings, we adopt DDQN as the backbone of TensorRL-QAS, -AS, leveraging its suitability for producing hardware-efficient ansatzes while avoiding redundancy with prior benchmarking work.
It is important to note, however, that DDQN may not be the optimal choice across all QAS-related tasks. For example, in Hamiltonian diagonalization or variational quantum classification, policy- gradient methods (e.g.. PPO, A2C) and actor-critic architectures often demonstrate better perfor- mance. Thus, while DDQN is especially well-suited for VQE-based ansatz construction in noisy- device settings, exploring alternative RL-agents for broader classes of QAS problems remains an interesting direction for future research.
E
QAS algorithms summary
E.1
CRLQAS: Curriculum reinforcement learning for QAS
The CRLQAS introduced 335 integrates curriculum-based reinforcement learning with quantum circuit architecture search (QAS) to address the challenges of variational quantum algorithms under hardware noise. In CRLQAS, the agent interacts with an environment where each state is a quantum circuit represented by tensor-based binary encoding. This 3D binary-encoding captures the gate operations across qubits, circuit depth (moments), and gate types (CNOT, RX, RY, RZ), allowing efficient processing by neural networks. The action space is defined by all possible single-qubit rotations and CNOT gates, amounting to 3.V + 2(N) actions for N -qqubit To reduce redundancy and improve learning efficiency, a mechanism for illegal actions is employed, which prevents the agent from appending consecutive identical gates the same qubit or repeating the same CNOT gate, effectively pruning the action space.
A central feature of CRLQAS is its curriculum learning mechanism, which adapts the difficulty of the optimization task as the agent learns. The curriculum governed by a moving threshold that determines when episode is considered successful. This threshold is not static; instead, it is
21

---

updated based on the agent's performance using a feedback-drives scheme. Initially, the threshold E set empirically, As the agent finds lower energies, the threshold is updated according Snew = | E2l + 6,
where &2 is the best energy found so far, d is a small amortization parameter (typically 0.0001), and Ja is the so-called fake minimum energy. The fake minimum energy is calculated as
lcab
where C2 are the coefficients in the Pauli decomposition of the molecular Hamiltonian. This value serves as a theoretical lower bound for the ground state energy, ensuring the agent is always guided towards physically meaningful solutions guaranteed by Rayleigh's variational principle.
The curriculum learning framework further incorporates adaptive rules: after every episodes (e.g.. G = 500), the threshold is greedily shifted to | {2/. and after every 50 successful episodes, decreased by 8/K (x 10). If the agent fails to improve the energy for 500 consecutive episodes, the threshold is reset to Snew + 5, allowing the agent to escape local minima. This dynamic adjustment ensures that the agent is neither stuck in an overly difficult regime nor allowed to stagnate at suboptimal solutions.
The agent's 's exploration-exploitttion trade-off is managed via an e-greedy policy, where the probability of taking random action decays exponentially from 1 to 0.05 as training progresses:
<(t) max 0.99995*).
Experience replay with a buffer size of 20, 000 and double Q-network 62 architectures with periodic target updates are used to stabilize learning. To encourage the discovery of compact circuits, CRLQAS employs a random halting scheme for episodes, where the maximum episode length is sampled from negative binomial distribution:
Pr(X niat) = -(Men - 1) p
1
p) PPai Paae
niaill
with Plact as the maximum number of actions and the failure probability. This approach increases the likelihood of shorter, more efficient circuits being discovered early in training.
For parameter optimization within each circuit, CRLQAS uses a hybrid Adam-SPSA optimizer. This combines the robustness of simultaneous perturbation stochastic approximation (SPSA) with the adaptive momentum of Adam, and employs a multi-stage shot budget strategy (from 100 to 10 shots) for efficient convergence in noisy environments. The noise simulation leverages the transerr matrix (PTM) formalism, with noisy gate operations precomputed offline and accelerated using JAX for GPU, yielding up six-fold speedup over conventional CPU-based simulations.
Empirically, CRLQAS achieves chemical accuracy (energy error below 1.6 x 10-3 Hartree) for molecules up to 8-qubit, often using fewer gates than baseline methods. The curriculum learning mechanism, driven by the fake minimum energy and adaptive thresholds, enables the agent to autonomously adjust the problem difficulty, maintain exploration, and efficiently find noise-resilient, gate-efficient quantum circuits even under realistic hardware noise profiles. This makes CRLQAS a robust approach for quantum architecture search in both simulated and hardware noisy intermediate- scale quantum settings.
E.2 TF-QAS: Training-free QAS
TF-QAS [42 is a novel quantum architecture search (QAS) method that aims to eliminate the need for desource-intensive circuit training. It primarily contains the following two steps: (1) Path-based proxy: The quantum circuit is represented by a directed acyclic graph [82]. Using the path count between the input-output nodes as a zero-cost complexity metric, this step filters unpromising circuits this enabling rapid pruning of the search space. (2) Expressibility proxy Evaluating the remaining circuit using Hilbert space convergence
(12) where (c( is the fidelity distribution of random states and Ph is Haar-random distribution. Although the progressive strategy balances efficiency in the first step and accuracy in the second, making it practical for NISQ-era quantum devices in Tab. [ we train L-nget in finding the ground state of 6-BEH, and consistently achieves 100-fold better accuracy across all seeds with more quantum quantum circuit, within smaller time as shown in Tab. 3
E(C) = DKL (Pc(F) || (F)),
22

---

Table TensorRL converges faster the optimal circuit structure than TF-QAS. The times noted for TensorRL are the average time taken by the TensorRL (trainable TN-init) and TensorRL (fixd TN-init) methods.
Method
Training time (h)
Time to solution (h)
TF-QAS
2.5
NA
6-BEHz
TensorRL-QAS
1.2
0.9
E.3 RA-QAS: Random agent for QAS
Following the methodology introduced in [37]. The random agent constructs quantum circuits by selecting actions uniformly random from the available gate set (RX, RY, RZ, CNOT) at each step of an episode. This process is independent of the current circuit state or any reward feedback.
After each randomly selected gate is appended to the circuit, all gate parameters (e.g., rotation angles) are initialized (new gates set to zero, previous gates retain optimized values), and the entire parameter set is optimized using classical optimizer (COB [83)) to minimize the VQE cost function. This global optimization ensures a fair comparison with learning-basee agents, which also rely classical optimization for parameter updates. Each episode consists of fixed number of steps D, corresponding to the maximum allowed circuit depth. The episode terminates early if the cost function C falls below a predefined threshold, ie the chemical accuracy, indicating successful diagonalization; otherwise, it proceeds until all steps are exhausted.
Algorithm 1 Random Agent for QAS
1: for episode = to Nepisodes do
2: Initialize empty quantum circuit C
3:
for step 1 to D do
4:
Randomly select action a from the gate set RX, RY, RZ Randomly select target (and control, if CNOT) qubits Append gate to circuit C
5:
6:
7:
Initialize parameters for new gate(s); retain previous optimized parameters Optimize all parameters Evaluate cost C2 for current circuit
8:
using COBYLA (fixed number of iterations)
9:
if C, < ( then break 12: end if
10:
11:
13: end for
14: Record metrics (success/failure,, gate count, circuit depth, absolute error)
15:
end for
As demonstrated in 337] the random agent's performance rapidly degrades with increasing problem size. The number of successful episodes (those that achieve the cost threshold) decreases sharply the number of qubits grows. Moreover, even in successful cases, the random agent typically produces longer and less efficient circuits compared to reinforcement learning- based approaches. This underscores the necessity of leveraging reward feedback and adaptive strategies for scalable and efficient quantum architecture search.
E.4 SA-QAS: Simulated annealing for QAS
The algorithm leverages principles from simulated annealing efficiently explore the space of possible quantum circuit architectures. The algorithm begins with an initial circuit populated with gates (identity gates). At each iteration t, we perform the following steps:
1. Randomly select a position in the circuit.
2. Replace the gate A, with the unitary matrix of randomly selected quantum gate.
3. Calculate the change in the loss function: AL, L, Lt-1-
23

---

4. Apply the acceptance criterion:
If AL < (improvement in loss), accept this change.
Otherwise, accept this change with probability e At//T, where T. = afi-
This process is repeated until either the loss remains stable for a predetermined number of itera- tions the maximum iteration limit is reached. The final sequence of gates , AM-1 is submitted as our solution. The algorithm employs a geometric cooling schedule T; = aT- The initial temperature To and the cooling rate are both configurable hyperparameters that control the exploration exxpoiitnneelliitttiinnnn trade-off. Higher initial temperatures encourage more exploration early in the search process, while the cooling rate determines how quickly the algorithm transitions to exploitation. The acceptance probability is defined as
P(accept) = {{ (e-AIu/T. otherwise
if AL <0
(13)
This probabilistic acceptance criterion is crucial for escaping local minima in the early stages of the search process, while gradually becoming more selective as the temperature decreases. SA-QAS terminates when either the loss function stabilizes (remains unchanged or changes below threshold) for a predetermined number of consecutive iterations or the maximum iteration limit M is reached. Upon termination, the sequence of gates Ag...., AM-1 represents our discovered quantum circuit architecture.
In our implementation, we maintain set of candidate quantum gates {RX, RY, RZ, CNOT) from which random replacements are drawn. The loss function quantifies how well the current circuit configuration performs the target task. The specific form of the loss function depends on the quantum computational task being addressed. The hyperparameters To and a significantly impact the search trajectory and should be tuned based on the specific problem characteristics.
F
Noise modeling
Depolarizing noise model In our simulations. we model gate errors using the depolarizing noise channel, which provides a simple yet effective description of incoherent noise in quantum circuits 84 For single-qubit state p, the depolarizing channel is defined as
{p) (1 -1- p)p
(14)
where p is the depolarizing error rate and / is the 2 x 2 identity matrix. This channel replaces the input state with the maximally mixed state I/2 with probability p. while leaving it unchanged with probability 1 - p. For multi-qubit systems, the depolarizing channel generalizes to
E(p) (1 p)p + 7
(15)
where n is the number of qubits. In our experiments, we amplify the I-qubit and 2-qubit gate error rates to 10-2 and 5 x 10-2, respectively, compared to currently available IBM Quantum [65] devices to assess the robustness of our algorithm. The depolarizing model particularly suitable for capturing the average effect of noise in circuits with many gates, as it treats all Pauli errors as equally likely and leads to a simple, analytically tractable form of noise 84]
Shot noise also known as samplesa noise, is an intrinsic source of statistical uncertainty in quantum circuit measurements, arising from the quantum nature of measurement and the discrete, probabilistic outcomes inherent to quantum mechanics [85] In quantum computing, the expectation values of observables are typically estimated by repeatedly executing a quantum circuit and recording the outcomes-a process referred to as collecting "shots". Each shot yields a single outcome sampled from the underlying quantum probability distribution, and the aggregate of many shots provides an empirical estimate of the expectation value. However, due to the finite number of samples, this estimate is subject to statistical fluctuations that scale as 1/VN, where N is the number of shots. This shot noise persists even in the absence of hardware errors and is governed by the Central Limit Theorem, which ensures that the distribution of the sample mean approaches a normal distribution as the number of shots increases [85). While increasing the number shots can reduce the variance of the estimate, practical constraints on quantum hardware often limit the feasible shot count, making shot noise a significant consideration in the design and analysis of quantum algorithms, especially for noisy intermediate-scon quantum devices.
24

---

G Performance of TensorRL-QAS under real hardware noise
To assess the practical relevance of TensorRL-QAS, evaluated quantum circuits discovered during RL training for 8 - H2O under realistic noise from JBMQ-Brisbang in Qiskit. We report the best episode (in this episode, we get the most compact circuit that achieves the ground state) and two circuits (random and random 2), randomly sampled from the pool of PQCs proposed by the RL-agent. Table@jissplas the noiseless and hardware-nowe error rates.
Table 4: Noiseless and noisy error for selected Senserrl-geeeertte circuits on 8 H2O; random (2) are PQCs randomly sampled from RL proposals. The best episode corresponds to the PQC with the smallest gate count.
Method
Noiseless error
BMMQ-Brisbane error
TensorRL, (fixed, best episode)
9.0 10
2.8 10-3
TensorRL (fixed, random I) TensorRL (fixed, random 2)
3.4 10 4
1.5 10
1.2 x 10-3
3.3 x 10-3
These results demonstrate the effectiveness TensorRL-QAS under realistic hardware noise and connectivity constraints. The inclusion of random PQCs sampled from the RL proposal pool further highlights the robustness of the architecture search, showing that typical agent-generated circuits maintain competitive performance. We remark that in general minimizing the noiseless error does not necessarily result in minimizing the noisy error and vice versa [35.
H Robustness of TensorRL
Table 5: TensorRL demonstrates robust performance by achieving ground state errors com- parable to baseline RL, and other non-RL methods for QAS, while constructing quantum circuits with fewer gates and shallower depth. Results are averaged over random seeds for finding the ground state of 8-CH20 and 10-CHyO. For 10-CH,O, the limited gate set RX, RY, RZ, CX prevents reaching the true ground state, indicating the need for more expressive gates (such in
ADAPT-VQE [16]..
Molecule
Method
Error
Depth
CNOT
ROT
Perform
8-CH,0
TensorRL (trainable TN-init, 2)
8.0 x 10
30
29
130
TensorRL (zero angle)
7.6 x 10
34
28
132
TensorRL. (fixed TN-init)
3.2 x 10-5
16
22
CRLQAS (rerun)
9.1 10
43
60
31
Vanilla
1.1 10
56
0.9
RA-QAS SA-QAS
1.3 x 10-3
64
107
53
0.8
7.9 x 10-3
35
59
28
10-CH,0
TensorRL (fixed TN-init)
4.5 10
15
37
TensorRL (trainable TN-init, 3)
4.3 x 10-3
39
43
163
0
TensorRI. (trainable TN-init, X StructureRI.
6.1 10-3
34
41
165
4.7 x 10
34
36
169
CRLQAS (rerun)
4.2 x 10
34
43
23
Vanilla RL
4.3 x 10
344
193
157
RA-QAS
6.7 x 10
179
354
120
0
SA-QAS
-3
38
23
H.1 More molecules
Table provides comparative analysis of TensorRL against both RL-based and non-RL-based quantum circuit ansatz discovery methods for finding ground states of 8-CH₂0 and 10-CH₂0 molecules. See Appendix, M.3 for the molecular configuration details. The results demonstrate that TensorRI. consistently achieves errors the same order of magnitude strong baselines (c.g., CRLQAS,
25

---

Vanilla RL), while often constructing quantum circuits with significantly fewer gates and reduced depth.
For 8-CHzO, the TensorRI. (fixed TN-init) variant achieves a notably low error (3.2 x 10-9) with the shallowest circuit (depth 16) and the fewest CNOT and rotation gates among all tested methods, highlighting its efficiency. For 10-CHy0, although none of the methods reach the true ground state due restricted gate set, TensorRL (fixed TN-init) still constructs the shallowest circuit (depth 15) with competitive error (4.5 x 10- outperforming other methods in circuit compactness. The experiments also reveal that methods relying solely on the RX, RY, RZ, gate set are limited in expressivity for larger molecules, suggesting the need for more expressive gates (as in ADAPT-VQE [16] to achieve ground state accuracy for challenging systems.
In summary, Table 5 highlights the robustness and efficiency of TensorRI, particularly its fixed TN-init variant, in constructing compact quantum eircuits while maintaining competitive accuracy compared to both RI. and non-RI. baselines. These results underscore TensorkL's potential for resource-efficient quantum eircuit discovery in quantum chemistry applications.
H.2 Reward signal analysis
Figure presents the cumulative reward as a function of training episodes for all evaluated methods: TensorRL. (trainable, 2), TensorRI. (fixed), StructureRI, Vanilla RI [21], and CRLQAS (35) (rerun). The æ-axis is shown on a logarithmic scale to highlight both early and late training dynamics.
Figure 4: Cumulative reward vs. episodes for all evaluated methods in the task of finding the ground state of 8-H20. All methods are trained for 48 hours, and the shaded regions represent the standard deviation across runs. TensorRL (trainable = 2) shows a sharp increase in accumulated reward compared to other methods, showing enhanced trainability. The shaded area in the plot corresponds to the standard deviation (o) from the average over random initializations of the neural network.
Reward vs episodes
TensorRL [troinable 2)
10000
TemsoRI (fowd)
Structurefi
Cumulative reward
5000
CRLOAS (rerus)
0
-5000
-10000
10
2.5 x 10
5 10
10
Episodes
TensorRL (trainable, = 2): This method demonstrates the strongest performance, with cumulative rewards increasing rapidly after approximately 2.5 10 episodes. By the end of training, it achieves the highest cumulative reward, exceeding 10, 000, and exhibits low variance, indicating stable and effective learning.
TensorRL (fixed): While this variant outperforms StructureRL and Vanilla RL, its cumula- tive reward plateaus at lower value compared to the trainable version. This suggests that trainable parameters in TensorRL are critical for achieving superior performance.
StructureRL. This method shows moderate improvement, with cumulative rewards remain- ing positive but not reaching the levels of either TensorRI. variant. The variance is also relatively contained, suggesting consistent, albeit limited, learning progress.
26

---

The baseline method struggles to achieve significant positive rewards, with its curve remaining near zero throughout training. This highlights the difficulty of the task and the necessity of structural enhancements.
Vanilla RL:
CRLQAS (rerun): This method initially suffers a sharp decline in cumulative reward, reaching values below - 10, 000 around 5 10 episodes. While some recovery is observed, the method remains substantially below the others final performance and exhibits high variance.
Overall, these results demonstrate that the proposed TensorRL (trainable) approach significantly outperforms baseline and alternative methods, both in terms of final cumulative reward and learning stability. The advantage of trainable structure within the RL framework is particularly evident, as fixed or less-structured baselines lag in both reward magnitude and convergence speed.
H.3 Success rate with scaling
Here we investigate Fig. 5 where we plot the probability of success of finding a parameterized quantum circuit that finds the ground state of 8- and 10-HoO molecules with an increasing number of episodes. The results are summarized in the following:
Figure 5:
The probability of finding successful episode i.e. quantum circuit that finds the ground state of 8-, 10-Hz0 consistently increases with TensorRL (fixed). The error bar represents maximum and minimum values across random neural network initializations, highlighting the robustness of TensorRL (fixed) even at its worst initialization compared to other methods.
8-H₂O
10-H>0
10
Probability of success
10-1
10-1
TenserRL (trainable, 2
10-2
10-
TensorfL (fixed
TensorRL fzero angls)
CRLQAS (sewu)
10-3
TinsceR (trainable, 3)
10-3
2
7
3
6
Intervals
Intervals
TensorRL (fixed) performance For 8-H₂O: Starts with low success (~0.11% () but steadily improves to approximately 50%. The performance curves shown represent maximum and minimum values over 5 random neural network initializations, demonstrating robust training stability. For 10-Hş0: While the average success probability remains around ~00% the maximum reaches an impressive 50% success rate for some initializations. This significantly outperforms other methods, which struggle to reach even 1% success at this qubit scale. Exhibits only ~5:1 performance reduction when scaling up from 8- 10-qubit. Preserves reasonable success rates at larger scales, making it valuable for practical applications.
CRLQAS comparison For 8-H₂0: Begins with extremely low success (~0.3%% and im- proves to ~10-20% with visible variability across different initializations. For 10-Hş0: Reaches only ~5% at its maximum performance across initializations, despite showing upward trajectory. Even at its best initialization, CRLQAS fails to approach the 50% maximum success rate achieved by TensorRL (fixed). Shows better scaling properties than trainable models but still substantially underperforms TensorRL (fixed) across all initializa- tion. Demonstrates potential for continued improvement with additional training, but lags significantly behind TensorRL (fixed) in maximum achievable performance.
27

---

Vanilla RL performance For O: Starts with moderate performance (~30%) but con- sistently declines across all initialization. For 10-HzD: Shows continuing performance degradation over time with success rates below 5% for all initializations, never approaching the 50% maximum achieved by TensorRI. (fixed). Ultimately falls below TensorRI. (fixed) for both system sizes at all initialization configurations. Highlights the limitations of non- tensor-based approaches for quantum circuit optimization, with even its best initialization performing poorly compared to TensorRL (fixed).
TensorRL. (fixed)'s 's average ~10% success probability for 10-qubit systems (with maxima reaching 50% across initializations) means up half of optimization episodes yield useful circuits in optimal configurations. This substantially reduces computational overhead for practical quantum simulations. No other method demonstrates this level of success at the 10-qubit scale, with most failing to exceed 1% success rate even at their best initialization. The constrained parameter space of fixed models avoids overfitting while maintaining expressivity. TensorRL (fixed) demonstrates the best capability for representing complex quantum correlations at scale, with its worst initialization often outperforming the best initializations of alternative methods. The consistent 50% maximum success probability for 10-qubit systems provides unprecedentedly efficient circuit discovery for moderately sized molecules, representing significant advancement for Hamiltonian ground state estimation.
H.4 TensorRL on CPU
Figure 6: Comparison of training time and circuit efficiency for TensorRL (fixed CPU/GPU) versus CRLQAS (GPU) and vanilla RL (GPU): TensorRL (fixed, CPU) achieves up to 10x fewer gates to chemical accuracy and matches surpasses GPU-based baselines in training time, enabling efficient and accessible quantum architecture search on standard CPUs. Here CA denotes the chemical accuracy.
Minimum gates CA
Time of execution
10
102.
TenerR, [Feo] (PU)
CRLOAS (CFU)
spticc
Tencrit. (trinable GPU)
Vanila RL (GPU)
per saccosdal specde
Chert,
10P.
Time (in second)
10
101
Erar
2
10-9
109
10a
10
10'
Teansft (fual GPU)
Temorff (eniaable GPU)
Trwselt (Feedd CRU)
CRLOAS (GPM))
Vania RL (GPU)
Number of gates
Our experimental results, summarized in Figure demonstrate that TensorRL in trainable mode on CPU achieves training efficiency that is competitive with, and often superior to, established baselines such as CRLQAS and vanilla RL operating on GPU, For quantum architecture search tasks up to 8-qubit, TensorRL (trainable, CPU) consistently matches or outperforms the GPU-based baselines terms of wall-clock time per episode and per successful episode. Notably, this improvement in computational efficiency does not come at the expense of solution quality: the minimum number of gates required to reach chemical accuracy comparable to, or slightly better than, those achieved by the baselines. These results establish that RL-QAS, when implemented with TensorRL, is practically trainable on standard CPUs for problem sizes up to 8-qubit, significantly broadening accessibility to high-quality quantum circuit design without reliance specialized hardware.
28

---

H.S
TensorRL-QAS beyond chemical Hamiltonians
Table presenns a comprehensive comparison of TensorRL and recent QAS methods on the 5-qubit Heisenberg model
Hilis
X + Y/4+1 2,2,+1
Zi:
(16)
and the 6-qubit transverse field Ising model (TFIM at h = 10
HTFIM
Σ ZiZ+1 + Σ X
(17)
1=1
These results demonstrate TensorRL's strong performance in general quantum many-body settings.
Table 6: Comparison of TensorRL-QAS with baseline QAS methods for the 5-qubit Heisenberg
and 6-qubit TFIM models. Bold: lowest gate count for competitive error. NA: not available.
Problem
Method
Error
Depth
CNOT
GATE
5 Heeieeberrgg
TensorRL (fixed)
7.6 x 10
15
5
33
TensorRL (trainable)
3.7 10 4
46
28
127
StructureRL
2.6 x 10-3
48
33
127
TF-QAS 42
1.2 10
NA
NA
35
DQAS (hest) 18
1.1 x 10
NA
NA
GQAS (hest)
86
7.1 x 10
NA
NA
35
GQAS SSL (best)
86
4.0 10
NA
NA
6-TFIM
TensorRL. (fixed)
x 10
13
TensorRI. (trainable)
1.8 x 10-6
29
16
111
StructureRT.
1.8 10
34
20
116
TF-QAS
1.4 x 10 12
NA
NA
36
TensorRL achieves competitive errors with much fewer gates than other approaches, especially for the fixed TN-init variant. For the 6-qubit TFIM, TensorRL finds solutions in circuits as small as 22 gates, an order of magnitude smaller than other competitive methods. The trade-off between solution accuracy and circuit complexity highlights TensorRL's capacity to generate highly compact and efficient circuits for quantum many-body systems outside chemical applications.
H.6 Scaling beyond 12-qubit
We examine the scalability of TensorRL-QAS and discuss its current limitations. To assess, we consider transverse field Ising models of 15-, 17-, and 20-qubit, with the Hamiltonian defined in Eq. 17 and magnetic field strength of 10-3 As established in Section 3 and Section 6 the TensorRL (fixed) variant yields the lowest computational overhead per reinforcement learning (RL) episode. Therefore, we adopt this method to evaluate scalability up 20-qubit across increasing system sizes and different Hamiltonians.
Our results, summarized in Table 7 demonstrate that TensorRL- yields accurate approximations of the ground state energy for the TFIM up to 17 qubits. Specifically, the method improves the energy by up to 21% % relative to the initial MPS state, achieving approximation errors of 4.4 10 for the 15-qubit system and 4.8 x 10-4 for the 17-qubi.. For the standard 20-qubit case with brickwork operator pool, while the absolute error remains low (6.7 x 10-4) the relative improvement is minimal (1%), indicating bottleneck due to limited action space and MPS bond dimension. Importantly, when the operator pool is expanded to include two-qubit operators (XX, YY, ZZ) to the already existing single-qubit rotations (RX, RZ, RY) and CX, the improvement on the same 20-qubit TFIM increases to 9%. This clearly demonstrates that more expressive set of actions enables TensorRL-QAS to scale more effectively and achieve superior ground state approximations for challenging large-scale problems. We believe this performance can further improved by utilising the operator pool described in ref. [16] or ref. (87).
29

---

Table Analysis of TensorRL-QQS for approximating ground-state energies of 15-, 17-, and
20-qubit transverse-field Ising models (TFIM) with weak magnetic field (h = 10 10-3). Results demonstrate up to % energy improvement over initial MPS states for systems up to 17-qubit (errors ~ 10-4), while the 20-qubit case reveals limitations due to restricted bond dimension (x = 2) and brickwork action space. With an extended operator pool for the 20-qubit case, substantial improvement is recovered.
Problem
Method
Error
Improvement
15-TFIM
TensorRL (fixed)
4.4 x 10
21%
17-TFIM
TensorRL (fixed)
10
19%
20-TFIM (bottleneck)
TensorKI. (fixed TN-init) TensorRI. (fixed)
6.7 10
1%
20-TFIM (actions: XX, YY, 22, RX, RZ, RY, CX)
6.1 10
9%
I Effect of different MPS to circuit mapping on TensorRL-QAS
To investigate how the properties of the tensor network (TN) initialization influence TensorRL-QAS we evaluated a range of TN-to-circui mappings by varying the number of TN layers (I to 3) as the warm-start for 6 2EH2, 8 H2O, and 10 H₂O molecular instances. For each configuration, recorded the RL-agent's success rate, gate counts, circuit depth, and ground-state error.
Table 8: Performance of TensorRI--QAS for different TN initialization depths and molecules. For each molecule, the lowest ground-state error for each group is bold.
Molecule
TN layers
Success rate
CNOT
ROT
Depth
Error
6 BEH₂
0.85
5
12
7
6.0 10
0.84
3
16
8
1.2 x 10
0.85
4
4
2.4 10-4
8 HgO
0.14
9
15
6
8.9 x 10
0.14
8
11
9
1.3 x 10
0.00
14
1.9 10
10 H₂O
0.13
15
17
7
4.1 10
0.11
13
8
9
5.8 x 10 10
0.00
18
12
16
1.8 10
For BEH2, both 2- and 3-layer initializations yield high success rates and competitive circuit costs. However, for 8 H₂O and 10 H20, increasing TN initialization depth to 3 layers consistently leads to RL failures, while the 2-layer variant maintains reasonable performance. These results indicate that deeper even within the same TN family, likely due to reduced learnability rather than overfitting. Our findings emphasize that effective TN initialization strikes practical balance between expressivity and learnability. Systematic exploration of fundamentally different TN structures and alternative mappings is valuable avenue for future work.
complex TN initializations
actually hinder overall algorithmic performance,
Performance of TN-compiled PQCs without RL refinement
To isolate the contribution of the TN-init alone, report ground-state energies for several molecular systems using only pure TN-compiled circuits, without any RL-based refinement. The table below shows the energy enror relative to the reference baseline.
This baseline highlights the importance of subsequent RL refinement achieving chemical accuracy, the pure TN-compiled circuits alone yield higher errors, especially for larger systems.
30

---

Table 9: Ground state energies and errors for pure TN-compiled PQCs (without any RL refinement). The error is relative to the reference ground-state energy.
Molecule
TN energy
True ground
Error
BEHz,
-4.885
-14.86
5.9 x 10
H₂O
73.29
-73.29
2.7 x 10-3
10 H₂O
-74.56
-74.56
4.8 x 10
LiH
-7.68
-7.87
1.9 10
K Scaling analysis: DMRG bond dimension vs RL-agent refinement
In this section we discuss how much the agent refines the performance of the tensor initialization with increasing bond dimension. Tab. [00 provides systematic comparison of RL optimization time (RL opt. time), time per episode and final accuracy (error) as function of the initial DMRG bond dimension for 12 LiH ground state preparation. The results reveal that increasing the bond dimension DMRG significantly reduces both the overall TensorRL training time and time per episode, as the higher-quality initialization leads to faster convergence and improved circuit efficiency.
Table 10: The RL optimization time and accuracy for 12-qubit LiH function of initial DMRG bond dimension. Increasing DMRG bond dimension leads to reduced RL training time and convergence chemical accuracy.
DMRG bond dim. (x)
RL opt. time (h)
Episode time (s)
Error
2
15.8
195
7.2 10
4
7.4
92
1.5 10
8
3.9
49
1.4 10-3
16
2.3
29
1.2
10
32
1.1
14
1.2 x 10-3
However, once the DMRG initialization reaches chemical accuracy, further improvements from RL become marginal, indicating that classical methods remain most efficient in this tractable regime. The practical advantage of the RL framework emerges as DMRG scalability breaks down, where quantum circuit construction and RL-based refinement become vital. This scaling analysis clarifies the operational regime of the TensorRL-QAS: as classical tensor methods approach their tractability limits, -uuddde quantum circuit discovery becomes increasingly relevant.
L
Scalability bottlenecks: Analysis of quantum simulator
The Fig. presents critical comparison between baseline reinforcement learning-assisted quantum architecture search (RL-QAS) and tensor network (TN) for RL-QAS (TensorRL-QASS) approaches, revealing fundamental scalability challenges quantum computing. As depicted the left panel, the baseline RL-QAS method exhibits severe computational bottleneck where statevector calculation time increases exponentially with both circuit depth and qubit count. For a system with N 12 qubits, computation time reaches approximately 10 seconds at circuit depths approaching 400, while even modest depths with N = 8 qubits require significantly less time (approximately 0.1 seconds at depth 200). This exponential scaling represents fundamental limitation to practical quantum applications.
A crucial consideration for real-world implementation is that reinforcement learning agents typically require multiple episodes to converge to optimal policies. training necessitates an average of 10,000 episodes, as is common in complex RL tasks, the total computation time for RL-QAS becomes entirely intractable. With each episode requiring up to seconds of statevector calculation for N = 12 qubits, the complete training process could require approximately 100,000 seconds (over 27 hours) of computation time solely for statevector calculations, This computational burden effectively precludes scaling to larger, more practical quantum systems where 12+ qubits would be required.
31

---

Figure 7: Time taken by Qulacs (88] simulator to get the statevector with baseline RL-QAS and TensorRL-QAS methods with varying numbers of qubits and depths of circuit. The substantial difference in computation time becomes more critical when considering that training requires approximately 10,000 episodes, making RL-QAS approaches intractable for larger qubit systems.
RL-QAS
TensorRL-QAA
10
N: 8
N: 10
N: 12
10
get
Time (s)
10-4
10-3
100
200
300
25
50
100
Depth of circuit
Depth of circuit
In stark contrast, the right panel demonstrates TensorRL-QAS's superior performance characteristics. TensorRL- -OAS requires substantially shallower circuits to achieve comparable results, with maximum depths of only about 100 compared 400 for baseline methods. More importantly, computation times remain under 0.5 seconds even for N = 12 qubits, reducing the projected training time for 10,000 episodes to approximately 5,000 seconds (less than hours). This dramatic improvement stems from TensorRL-QAS's innovative "warm start" approach, which leverages matrix product states (MPS) for parameterized quantum circuit (PQC) initialization.
The warm start strategy enables TensorRL-QAS to converge to solutions in significantly fewer steps, creating a computational advantage that increases with qubit count. By effectively mapping the tensor network representation to quantum circuit parameters, TensorRL-QAS takes step forward to addresses one of the primary obstacles in quantum architecture search i... scalability. This initialization method bypasses the exponential growth in statevector computation time that plagues conventional approaches, making TensorRL-QAS substantially more suitable for practical quantum applications requiring larger qubit systems.
The logarithmic scale in the figure clearly illustrates the magnitude of this advantage, with the baseline method showing steep growth curves across all qubit counts. At the same time, TensorRL-QAS maintains relatively modest computation times even as system size increases. These results strongly suggest that tensor network initialization strategies represent promising direction for overcoming limitations in quantum machine learning approaches, potentially enabling quantum advantage in practical applications that would otherwise remain computationally intractable using conventional methods.
M Experimental setup
M.1 Reward function
We employ the reward function R from
21 35
with modifications for quantum tasks:
5,
if if <
-5,
if &
R
(18)
C--1
C.
max
-1)
otherwise
C,-1
Cmin
32

---

where C2 = (O) represents the variational energy at step t, is precision threshold,
and T. denotes the episode's 's step limit. The =5 rewards terminate episodes upon success (energy 5) or failure (timeout with energy 2 Q)
For quantum chemistry tasks, the threshold is typically set 1.6 10-3 it defines the precision such that realistic chemical predictions can be made. The goal of the agent is to obtain an estimated value of Cmin within such precision. Whereas, while solving the Heisenberg model, consider to 10 and for the transverse field Ising model we set it as 10 2, which are obtained by running SA-QAS 39 on these Hamiltonians.
M.2 Number of steps per episode
Here we report the number steps (i.e., maximum number of gates can be added by the RL-agent) per episode for TensorRL-QAS and baseline QAS algorithms reach convergence to the ground state energy, as summarized in Table I1 The results highlight that VeensoRL--QAS consistently
Table 11: Steps per episode required for convergence ground energy.
Qubits
TensorRL-QAS
CRLQAS
RAQAS
6
20
70
97
8
20
250
300
10
20
350
477
12
40
450
450
15
60
NA
NA
17
60
NA
NA
20
80
NA
NA
achieves convergence with dramatically fewer steps, often by a factor 3 5x or more compared to CRLQAS and RAQAS, across all tested system sizes. This reduction in episode length reflects the benefit of starting from a physically motivated tensor network initialization, substantially accelerating learning and improving computational efficiency in quantum circuit discovery.
The geometry and basis of molecules used in this research. The coordinates are in Angstrom units.
Table 12:
Molecule
Geometry
Basis
6-BEH₂
H (0,0,-1.33); Be (0,0,0); (0,0,1.33)
STO-3G
8-H20
H (-0.02,-0.0) O (0.44.044550 (1.48,
STO-3G
8-CH,0
C (1000);)(((00000))
STO-3G
10-Hş0
(4.02--0000 (1.4..0....
6-31G
10-CHş0
(0..0.0): H (-0.23,1.06.0)
STO-3G
12-LiH
Li (0,0,0): H (0.0.3.40)
STO-3G
M.3 Molecular configuration
Using the Born- Oorn-Oppenheimer approximation, fix finite basis in case STO-3G, for all the molecules except 10-H20, where we use 6-31G to extend the number of orbitals and to discretize the system. We choose the molecules based on the top 20 molecules provided by Pennylane [61], and throughout the paper, we use Pennylane along with OpenFermion to generate the chemical Hamiltonians in Tab. 12
M.4
MPS to PQC conversion gate resources
In Tab. 13 we present the resources in the one-depth brickwork structure required when converting from MPS to parameterized quantum circuit. The resources quantified by the number of CNOTs,
33

---

rotations (ROT) and depth (Depth) of the circuit and in the case of 10-qubit, the bond dimension All the circuits are simulated using Qulacs 88
Gate counts and depth for MPS-to-cireuit conversion across different system sizes. The number of gates increases with bond length X-
Table 13:
Qubits
CNOT
Depth
ROT
5 Ix 2)
12
27
75
6 6 2)
15
27
93
8(x
21
27
129
10 (X = 2)
27
165
10 (X = 3)
26
27
160
12 (x 2)
33
27
201
15 2)
42
27
255
(x = 2)
48
20 X = 2)
57
345
N Computed resources
We utilize a system with an AMD Rome 7H12 CPU based on AND Zen architecture and an Nvidia Ampere A100 GPU. The time of training and approaching the first and minimum error is presented Tab. 14] all times are noted hours. For all the configurations, we utilized 2 CPUs, each with maximum 4GB memory and maximum 40 GB GPU memory.
Table 14: The training time for different TensorRL-QAS methods across various qubits. We notice that TensorRI. (fixed) converges much faster the desired error than other methods.
Method Metric (in hours)
5q
6q
8q
10q
12q
TensorRL (fixed)
Time to min eтror
5.88
0.05
0.91
11.22
0.58
Time first success
5.24
0.05
0.15
0.53
0.58
Total train time
6.42
0.13
1.48
34.82
48.00
TensorRL (trainable)
Time to min error
6.14
0.24
27.60
0.56
48.15
Time to first success
6.14
0.02
0.40
0.56
17.07
Total train time
29.84
2.00
40.84
47.50
48.00
StructureRL
Time to min error
5.42
0.66
35.47
23.92
14.86
Time to first success
5.42
0.27
1.03
23.92
4.81
Total train time
29.84
1.91
45.75
47.69
48.00
Borderline impact
Accelerating quantum scientific discovery (positive): TensorRL-QAs enables scalable and efficient quantum circuit design, which can accelerate research in quantum chemistry. materials science, and related domains. This may facilitate the discovery of new molecules, drugs, or materials, with broad societal benefits in medicine, energy, and technology.
2. Democratizing access to quantum algorithm development (positive): By significantly reducing computational overhead and enabling CPU-only training for sizable quantum systems, our method lowers barriers entry for researchers and institutions with limited resources, promoting broader participation in quantum computing research.
3. Enabling practical quantum advantage (positive): The approach addresses key bottle- necks hindering the realization of quantum advantage near-term hardware. Making
34

---

variational quantum algorithms more feasible and robust under noise brings practical quan- tum applications closer reality, with potential for positive economic and technological impact.
4. Potential for misuse (negative); As with any advancement in quantum computing, more efficient quantum circuit discovery could be leveraged for malicious purposes, such as breaking cryptographic protocols or enabling unauthorized access to secure systems. While this work focuses on quantum chemistry and physics applications, the generality of the approach means it could be adapted for less benign purposes.
5. Environmental considerations (negative): Although the method reduces computational requirements relative to previous approaches, quantum computing research-especially scale-remains energy-intensive. Widespread adoption could contribute to increased energy consumption if not managed responsibly, particularly quantum hardware becomes more prevalent.
Safeguards and limitations: The code is released openly with documentation at https://ittuu..co///Aaa cwt.ccom////ceae wwwp///gsss//eennrRRR..A... and appropriate licensing. The work does intro- duce high-risk assets such as generative models for disinformation or surveillance, Limitations and future work discussed, including the need for real-device validation and further exploration of action space expressivity; immediate risks are assessed to be low in the context of the presented research. This work discusses both the potential positive and negative societal impacts, in line with NeurlPS requirements for a balanced and responsible broader impact statement.
35


Reviews:
[{"review_id": "iqaO6aiWvB", "reviewer": "Reviewer_XvsW", "summary": "This paper presents TensorRL-QAS, a framework that makes finding optimal quantum circuits more efficient and scalable with reinforcement learning methods. The core idea is to combine classical tensor networks with RL. First, an approximate solution is found using DMRG and converted into an initial quantum circuit. An RL agent then takes this \"warm-start\" circuit and intelligently adds more gates to improve it. By starting with a good initial guess, the search for the best circuit is significantly sped up. The paper shows that this method creates much more compact and efficient circuits (with up to 10x fewer CNOT gates) and drastically reduces computation time (by up to 100x) compared to previous approaches. The framework proves effective and robust in both noiseless and noisy simulations, making it a promising tool for near-term quantum computing.", "questions": "Is it possible to cooperate the proposed method with real hardware noise model?\nHow would the \"fixed TN-init\" model, which relies on preparing an initial statevector, be implemented in practice on current hardware? This step seems non-trivial and may introduce significant overhead not accounted for in the simulations.\nHow does the performance of TensorRL-QAS change if a different TN-to-circuit mapping is used, or if a different tensor network structure is used for the warm-start?", "limitations": "see above", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The proposed method from this paper is novel and show significant improvement by doing rigorous comprehensive experiments comparing to multiple RL-based and non RL-based methods. Also, this paper provide codebase for reproducibility. \nOne of the limitation of this paper is that their experiments are based on simulations rather than execution on actual quantum hardware. Also, it would be better to state that whether the choice of different initial structure of MPS could affect the final outcome.", "was_revised": true, "final_justification": "I generally interested in the paper from the start, and authors mostly addressed my comment, also the comment from Reviewer e9Vh about dmrg also seems addressed by authors. Thus, I keep stand on positive score.", "rebuttal": "We sincerely thank the reviewers for their thoughtful and encouraging feedback. We greatly appreciate your recognition of the novelty of our proposed method, the comprehensive and rigorous nature of our experimental comparisons across RL-based and non-RL-based baselines, and our efforts in providing a reproducible codebase for the community. In the following we address your questions and suggestions.\n\n### **Is it possible to cooperate the proposed method with real hardware noise model?**\n\nThank you for this valuable suggestion. We fully agree that validating the proposed method on real hardware is crucial for demonstrating its practical applicability. Due to current limitations, most notably the long queue times associated with available QPUs, our primary studies focus on simulations using realistic noise models. To ensure robustness, we also evaluated the method under elevated noise levels (5x and 10x the baseline) to mimic harsher hardware conditions, as discussed in [Patel, Yash J., et al., \"Curriculum reinforcement learning for quantum architecture search under hardware errors,\" ICLR (2024)]. As shown in that work, simulated noise models often yield trends consistent with actual QPU runs.\n\nTo further respond to your point, we conducted an additional experiment: we selected several randomly chosen successful RL-generated circuits (i.e., those achieving chemical accuracy) for 8-$H_2O$ found in noisy simulations (using depolarizing noise), and ran these circuits on $\\texttt{IBMQ-Brisbane}$ using $\\texttt{Qiskit}$. The results are shown in the table below:\n\n| Method                                      | Noiseless error              | Noisy error (IBMQ-Brisbane)      |\n|--------------------------------------------|-----------------------------|-----------------------------------|\n| TensorRL (fixed TN-init) (best episode)    | $9.0\\times10^{-4}$           | $2.8\\times10^{-3}$               |\n| TensorRL (fixed TN-init)  (random 1)       | $3.4\\times10^{-4}$           | $\\mathbf{1.5\\times10^{-3}}$      |\n| TensorRL (fixed TN-init)  (random 2)       | $1.2\\times10^{-3}$           | $3.3\\times10^{-3}$               |\n\nThese findings further demonstrate the practical relevance of TensorRL-QAS and support the observation from [Patel et al., ICLR 2024] that the best-performing circuit in simulation may not always be the best on real quantum hardware. This emphasizes the importance of evaluating candidate architectures directly on QPUs when possible and motivates our future work toward a tighter integration with specific hardware noise profiles. We will include this insights and results in the **appendix** of the revised manuscript.\n\n### **How would the \"fixed TN-init\" model, which relies on preparing an initial statevector, be implemented in practice on current hardware? This step seems non-trivial and may introduce significant overhead not accounted for in the simulations.** \n\nThank you for raising this important point regarding the practical implementation of the \"fixed TN-init\" model, which relies on preparing an initial statevector.\n\n1. **Compiling TN-init to QPU:** In practice, we would compile the TN-init (tensor network-initialized) state into a quantum circuit using gates natively supported by the target hardware. This allows us to adapt the initialization procedure to the device-specific gateset while ensuring physical realizability.\n\n2. **Maintaining TN-init on QPU:** If the compiled circuit for the TN-init differs in fidelity from the ideal statevector because of noise, we can employ error mitigation strategies, such as tensor error mitigation [Filippov, Sergei, et al. arXiv preprint arXiv:2307.11740 (2023).], to bring the prepared state as close as possible to the ideal TN-init.\n\n3. **TensorRL atop the TN-init:** With the TN-init physically implemented as a compiled circuit and error mitigation in place to counteract hardware noise, we can then apply the TensorRL using this fixed initialization. The trained policy and subsequent circuit evolutions can thus be performed atop a stable, experimentally feasible TN-init, while continuing to apply mitigation strategies to maintain fidelity throughout the computation.\n\nOverall, this approach is practically feasible, clearly at the cost of using error mitigation techniques —coming with their unavoidable overheads— to counteract hardware noise. *Due to current limitations in accessing QPUs and limited time on GPUs from our side, a thorough exploration and experimental demonstration of this workflow is left as future work*.\n\n### **How does the performance of TensorRL-QAS change if a different TN-to-circuit mapping is used, or if a different tensor network structure is used for the warm-start?**\n\nThank you for raising this important question. Due to time constraints, we were not able to comprehensively investigate alternative TN-to-circuit mappings or entirely different tensor network (TN) structures. However, we sought to probe related effects by varying the number of TN layers in our initialization (from 1 to 3 layers), which closely emulates changes in TN structure and, implicitly, circuit entanglement patterns. For each configuration, we measured the success rate, as well as circuit and error statistics for 6-, 8-, and 10-qubit molecules. The proposed RL agent circuit performance and parameters are summarized in the table below:\n\n| Qubit count | TN layer | Success rate | CNOT | ROT | Depth | Error     |\n|-------------|----------|--------------|------|-----|-------|-----------|\n| 6           | 1        |  0.8478      | 5    | 12  | 7     | **0.00006**   |\n|             | 2        |  0.8418      | 3    | 11  | 5     | 0.00012   |\n|             | 3        |  0.8486      | **2**| **4**| **4**| 0.00024   |\n| 8           | 1        |  0.1423      | 9    | 15  | **6** | **0.00089**   |\n|             | 2        |  0.1398      | **8**| **11**| 9   | 0.00133   |\n|             | 3        |  0.0000      | 6    | 14  | 7     | 0.00196   |\n| 10          | 1        |  0.1288      | 15   | 17  | 17    | **0.00041**   |\n|             | 2        |  0.1097      | **13** | **8** | **9** | 0.00058   |\n|             | 3        |  0           | 18   | 12  | 16    | 0.00178   |\n\nFor 6 qubits, both 2- and 3-layer TN initializations perform well, achieving comparably high success rates and competitive circuit costs. However, for 8 and 10 qubits, increasing the TN initialization depth to 3 layers leads to RL failures (resulting in zero successful episodes), whereas the 2-layer initialization maintains reasonable performance. This indicates that deeper or more complex initial circuits, even within the same TN family, can hinder, rather than artificially boost, the overall algorithmic performance. Thus, our findings suggest our method’s success is not attributable solely to an over-optimized or \"overfitted\" warm-start structure. Instead, the chosen TN initialization achieves a practical trade-off between expressivity and learnability.\n\nWhile a systematic exploration of alternative TN-to-circuit mappings and fundamentally different TN structures remains a valuable direction for future work, our present experiments with varied TN layers serve as an initial, closely related probe. We will include these additional results and analyses in the **appendix** of the revised manuscript.\n\nWe hope our response properly addresses all your questions and suggestions. Kindly let us know if you have any more questions and/or comments."}, {"review_id": "Nt9lu3DeLB", "reviewer": "Reviewer_D2Gn", "summary": "The paper combined three proposed ingredients into the novel quantum-architecture search pipeline : (i) DMRG-based tensor-network ground-state preparation, (ii) an MPS-to-quantum-circuit compiler, and (iii) reinforcement-learning optimization using Double-DQN agent to find additional gates that better improve the accuracy. Empirical results on simulated quantum-chemistry Hamiltonians show improved convergence (time-to-accuracy and total gate count) over a baseline RL-QAS without warm starts.", "questions": "Can the proposed method be extended to larger systems beyond 12 qubits?\n\n\nWhat is the challenge to present practical experimental results using the proposed method?\n\n Why is RL especially useful for the given task?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1) This paper combined tensor networks with reinforcement learning to tackle quantum architecture search, which is a highly interesting and promising direction. TensorRL-QAS achieves up to a 10-fold reduction in CNOT count and circuit depth compared to baseline methods, while maintaining or even surpassing chemical accuracy.\n\n2) The manuscript is well-written and the three-stage workflow is easy to follow; implementation details, e.g., network sizes, optimizer hyper-parameters, simulator settings are explicit, which facilitate easy reproduction.\n\n3) On different molecules tested in Table 1, the proposed method consistently accelerates learning and shortens circuits relative to the baseline methods.\n\n4) Simulation results on both noiseless and noisy situations verify the robustness of this method.\n\n \nWeakness and suggestions for improvement\n\n1) The simulations are limited to 12 qubits, whereas the referenced work [12] reports results on systems with up to 100 qubits. A systematic scaling study on larger qubit numbers would significantly strengthen the paper and demonstrate the broader applicability of the proposed method.\n\n\n2) The authors have demonstrated some results about noisy cases, while it would be useful to provide some experimental results on real quantum device to demonstrate its practical use.\n\n \n3) Since the method includes three parts, the author might implement a more detailed ablation study to investigate the contributions of all the three parts. For example, the authors may consider to include the results of a pure TN-compiled circuit without RL refinement.\n\n4）It would be useful to broaden the baseline pool: add recent evolutionary-search and gradient-based QAS methods.", "was_revised": true, "final_justification": "I feel the work includes interesting contributions as I mentioned in my review comments. I agree with the other reviewers especially regarding the scalability. My final score is based on these contributions of this paper and its limitation on scalability and applicability,", "rebuttal": "Thank you very much for your thoughtful and positive feedback. We are pleased that you find TensorRL-QAS interesting and promising. Moreover we appreciate your positive remarks on the clarity, reproducibility, and robustness. In the following we systematically address all your questions and suggestions.\n\n### **Can the proposed method be extended to larger systems beyond 12 qubits?**\n\nThank you for the question. In the current article in appendix we show that TensorRL can be utilized upto 20 qubits, we present these results explicitly in the table below:\n\n| Problem                 | Method                                  | Error           | Improvement |\n|-------------------------|-----------------------------------------|-----------------|-------------|\n| 15-TFIM                 | TensorRL (fixed TN-init)                | 4.4 × 10⁻⁴      | 21%         |\n| 17-TFIM                 | TensorRL (fixed TN-init)                | 4.8 × 10⁻⁴      | 19%         |\n| 20-TFIM (bottleneck)    | TensorRL (fixed TN-init)                | 6.7 × 10⁻⁴      | 1%          |\n| **20-TFIM (extended action space: *XX*, *YY*, *ZZ*, *RX*, *RZ*, *RY*, *CX*)** | TensorRL (fixed TN-init)                | 6.1 × 10⁻⁴                | **9%**      |\n\nThe improvement is over the TN initialization energy.\n\n### **What is the challenge to present practical experimental results using the proposed method?**\n\nThe main challenge of our current approach is the computational bottleneck in the RL training phase. More specifically, each episode requires simulating and optimizing a parameterized quantum circuit, and computing its expectation value multiple times (equal to the number of steps i.e. the maximum depth of the circuit set as a hyperparameter). Already for ~12 qubits, this becomes computationally intensive due to the number of episodes required for convergence/solving the task and the increasing number of query to the quantum simulator.\n\n### **Why is RL especially useful for the given task?**\n\nWe thank the reviewer for this valuable question. To show the effectiveness of the RL agent we conduct an experiment where instead of utilizing an RL algorithm to choose and optimize quantum gates we use a hardware-efficient ansatz (HEA) after the fixed TN-init (namely **TN-VQE**). Each gate of the HEA is explicitly optimized by COBYLA optimizer individually, the results are summarized below:\n\n| Molecule      | Method                                      | Error                | Depth  | CNOT   | ROT      |\n|---------------|---------------------------------------------|----------------------|--------|--------|----------|\n| 6-BEH₂        | **TensorRL (fixed TN-init)**                | 6.8×10⁻⁵            | **7**  | **5**  | **12**   |\n|               | TensorRL (trainable TN-init)                | 6.0×10⁻⁵            | 33     | 22     | 96       |\n|               | TN-VQE                                      | 5.4×10⁻³            | 10     | 8      | 12       |\n| 8-H₂O         | **TensorRL (fixed TN-init)**                | 8.9×10⁻⁴            | **6**  | **9**  | **15**   |\n|               | TensorRL (trainable TN-init)                | 2.0×10⁻⁴            | 36     | 30     | 146      |\n|               | TN-VQE                                      | 2.7×10⁻³            | 10     | 14     | 6        |\n| 10-H₂O        | **TensorRL (fixed TN-init)**                | 4.1×10⁻⁴            | **17** | **15** | **17**   |\n|               | TensorRL (trainable TN-init, χ=3)           | 6.7×10⁻⁴            | 33     | 34     | 168      |\n|               | TensorRL (trainable TN-init)                | 7.1×10⁻⁴            | 35     | 48     | 173      |\n|               | TN-VQE                                      | 1.1×10⁻³            | 26     | 51     | 14       |\n| 12-LiH        | **TensorRL (trainable TN-init)**            | **1.0×10⁻²**        | 31     | 37     | 203      |\n|               | TensorRL (fixed TN-init)                    | 2.4×10⁻²            | **15** | **30** | **9**    |\n|               | TN-VQE                                      | 8.6×10⁻²            | 33     | 81     | 29       |\n\nThis reveals that without an RL training we can optimize the DMRG-initialized circuit to a certain extent but it consistently underperforms the RL-based approaches, both in terms of solution accuracy (error) and circuit efficiency (depth, CNOT, and ROT counts). Notably, for the 6-, 8-, and 12-qubit molecules, TN-VQE is unable to find the ground state with comparable accuracy to the RL-based methods, and in some cases fails to solve the problem altogether. This solidifies the fact that having a high-quality initialization alone is insufficient; the RL framework provides a substantial advantage by better guiding the circuit construction and parameter optimization. We will add these insights in the **Results** section in the revised version of the article.\n\n### **The authors have demonstrated some results about noisy cases, while it would be useful to provide some experimental results on real quantum device to demonstrate its practical use.**\n\nThank you for this valuable suggestion. We agree that experimental validation on real quantum hardware is important for demonstrating practical relevance. Due to current limitations, primarily long queue times for available QPUs, we focused our main studies on simulations with realistic noise models, including tests with noise levels elevated by 5 and 10 times to probe the robustness of TensorRL-QAS. As shown in [Patel, Yash J., et al., \"Curriculum reinforcement learning for quantum architecture search under hardware errors,\" ICLR (2024)], simulated noise models generally align closely with results from real QPUs. To further address your point, we also evaluated randomly chosen successful (which passes the chemical accurayc) RL circuits generated from noisy simulation with depolarizing noise for 8-$H_2O$ and then run the circuit on $\\texttt{IBMQ-Brisbane}$ with $\\texttt{Qiskit}$; the results are displayed below which further supporting the practical applicability of TensorRL-QAS.\n\n| Method                         | Noiseless error         | Noisy error (IBMQ-Brisbane)          |\n|---------------------------------|--------------------------|---------------------------|\n| TensorRL (fixed TN-init) (best episode)        | $9.0\\times10^{-4}$ |    $2.8\\times10^{-3}$   |\n| TensorRL (fixed TN-init)  (random 1)      | $3.4\\times10^{-4}$ |    $\\mathbf{1.5\\times10^{-3}}$   |\n| TensorRL (fixed TN-init)  (random 2)     | $1.2\\times10^{-3}$ |    $3.3\\times10^{-3}$   |\n\nThese results solidifies one of the claim in [Patel, Yash J., et al., \"Curriculum reinforcement learning for quantum architecture search under hardware errors,\" ICLR (2024)] that the best performing circuit in simulated scenario might not be the best performing in QPU.\n\n### **...the authors may consider to include the results of a pure TN-compiled circuit without RL refinement.**\n\nThank you for your suggestion regarding a more detailed ablation study. In response, we provide a table of ground-state energies using only pure TN-compiled circuits without RL refinement (TN-energy) for several molecular systems. The error from the reference (true energy) energies serves as a baseline for our method, isolating the contribution of TN compilation alone.\n| Problem | TN-energy | True energy | Error |\n|--------|--------------------|--------------------------|-----------------------|\n| 6-BeH₂    | -14.8556       | -14.8616      | 0.00596              |\n| 8-H₂O          | -73.2914       | -73.2941       | 0.0027              |\n| 10-H₂O         | -74.5633       | -74.5681       | 0.00479              |\n| 12-LiH         | -7.6808        | -7.8746       | 0.19378              |\n\n\n### **It would be useful to broaden the baseline pool: add recent evolutionary-search and gradient-based QAS methods.**\n\nThank you for the valuable suggestion. In the revised version of the manuscript we added a gradient-based QAS (GQAS) [He, Zhimin, et al. Neural Networks 179 (2024): 106508.] and DQAS [Zhang, S. X., Hsieh, C. Y., Zhang, S., & Yao, H. (2022). Quantum Science and Technology, 7(4), 045023.] in the table below. This will be added in the **appendix**. Unfortunately for evolutionary-based QAS we could not find any benchmarks that is in line with our optimization method that is why we cite the reference [Zhang, Anqi, and Shengmei Zhao. arXiv preprint arXiv:2212.00421 (2022).] in the **Related works**.\n\n| Problem      | Method                                  | Error              | Depth | CNOT | GATE |\n|--------------|-----------------------------------------|--------------------|-------|------|------|\n| 5-Heisenberg | TensorRL (fixed TN-init)                | $7.6 \\times 10^{-2}$   | 15    | 5    | **33**   |\n|  | TensorRL (trainable TN-init, $\\chi=2$)  | $3.7 \\times 10^{-4}$   | 46    | 28   | 127  |\n|  | StructureRL                             | $2.6 \\times 10^{-3}$   | 48    | 33   | 127  |\n|  | TF-QAS                                  | $1.2 \\times 10^{-3}$   | NA    | NA   | 35   |\n|  | DQAS (best)                             | $1.11 \\times 10^{-1}$  | NA    | NA   | 35   |\n|  | GQAS (best)                             | $7.06 \\times 10^{-4}$  | NA    | NA   | 35   |\n|  | GQAS-SSL (best)                         | $\\mathbf{4 \\times 10^{-6}}$     | NA    | NA   | 35   |\nwhere **NA** denotes results are not available.\n\nWe hope that our response states all the concerns and questions raised by you. Kindly let us know if you have any more suggestions and questions."}, {"review_id": "zKmLpPQ3R5", "reviewer": "Reviewer_AdE9", "summary": "The paper introduces TensorRL-QAS to address the scalability challenges in Quantum Architecture Search (QAS). Specifically, the authors first employ the classical Density Matrix Renormalization Group (DMRG) algorithm to obtain a good Matrix Product State (MPS) approximation of the target state. This MPS is then mapped into a Parameterized Quantum Circuit (PQC) to serve as a \"warm-start\" initialization. Finally, a Reinforcement Learning (RL) agent sequentially adds quantum gates to this initial circuit to further optimize its structure and approach the true ground state.\n\nThis \"TN warm-start\" strategy significantly narrows down the RL search space, leading to a substantial reduction in the number of function evaluations and training time. Experimental results on molecular simulation problems up to 12 qubits demonstrate that this method, compared to baseline approaches that start from scratch, can generate more compact circuits with lower or comparable errors, showing superiority in both noiseless and noisy environments.", "questions": "1. Your method benefits from a high-quality initial circuit generated by DMRG. To better isolate the value of RL, could you add a \"TN-VQE\" baseline? This would involve using the COBYLA optimizer directly to optimize the newly added gates, without any RL framework. This would help distinguish the contributions of \"a good starting point\" versus \"a good search strategy\".\n2. Mapping the MPS to a single-layer brickwork circuit is a strong assumption. How much does this choice affect the final performance? Have you explored other mapping schemes, such as generating deeper or differently structured initial circuits? Could the current results be overstated due to this highly optimized initial structure?\n3. In the fixed TN-init model, the initial circuit parameters are frozen. The RL agent's task is essentially to find an optimal solution for a new, shallower circuit. How does the search difficulty of this new problem compare to the original one?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The problem addressed—the scalability bottleneck of RL in QAS—is a core challenge in the current Noisy Intermediate-Scale Quantum (NISQ) era.\n2. Combining the physical insights of tensor networks (specifically DMRG/MPS) with reinforcement learning is a novel approach.\n\nWeaknesses:\n1. The performance boost of TensorRL-QAS is likely attributable to its high-quality starting point from DMRG, rather than the superiority of the RL agent's search strategy. The paper conflates the benefits of a \"better initialization\" with those of a \"better search algorithm.\" A fairer baseline would involve: 1) Directly optimizing the DMRG-generated initial circuit with a classical optimizer (like COBYLA) without RL. 2) Having all methods start from the same ansatz to see which RL strategy finds the optimal structure more effectively.\n2. A core promise of QAS is to discover novel, counter-intuitive, and efficient quantum circuit structures. However, this work maps the MPS to a single-layered brickwork structure, which is a very strong constraint. While hardware-friendly, it drastically shrinks the search space, potentially defeating the purpose of QAS. The claimed \"scalability\" might be a byproduct of this extreme space simplification. The mere 1% improvement for the 20-qubit system in Appendix E.6 suggests this MPS structure may struggle to represent entanglement in more complex systems, casting doubt on its generality and true scalability.\n3. The best-performing model, TensorRL (fixed TN-init), uses a fixed initial circuit, leaving very limited room for RL to operate. For such a \"fine-tuning\" task, is a complex RL framework necessary? Why not simply append a few layers of a standard ansatz to the initial circuit and optimize all parameters at once with a classical optimizer? The paper lacks a discussion of this simpler alternative, making the necessity and contribution of RL questionable.\n4. The title argues for scalability, yet the experiments are demonstrated up to 20 qubits. In the current stage, \"scalable\" is more appropriately used for systems approaching 50+ qubits, making the claim seem premature.", "was_revised": true, "final_justification": "If the authors commit to thoroughly revising the scalability claims across the entire paper to better reflect their experimental scope, I am willing to raise my score to borderline accept.", "rebuttal": "Thank you very much for your thoughtful feedback. We appreciate your acknowledgment of the novelty in integrating tensor network methods and reinforcement learning. In the following we systematically address your concerns:\n\n### **...To better isolate the value of RL, could you add a \"TN-VQE\" baseline? This would involve using the COBYLA optimizer directly to optimize the newly added gates, without any RL framework...**\n\nThank you for your insightful suggestion. In response, we have added the \"TN-VQE\" baseline to our results table, as shown below. This baseline utilizes the DMRG-initialized circuit as a starting point and employs the classical COBYLA optimizer to optimize each newly added gate, without RL framework.\n\n| Molecule      | Method                                      | Error                | Depth  | CNOT   | ROT      |\n|---------------|---------------------------------------------|----------------------|--------|--------|----------|\n| 6-BEH₂        | **TensorRL (fixed TN-init)**                | 6.8×10⁻⁵            | **7**  | **5**  | **12**   |\n|               | TensorRL (trainable TN-init)                | 6.0×10⁻⁵            | 33     | 22     | 96       |\n|               | $\\vdots$                            | $\\vdots$        | $\\vdots$     | $\\vdots$    | $\\vdots$       |\n|               | TN-VQE                                      | 5.4×10⁻³            | 10     | 8      | 12       |\n| 8-H₂O         | **TensorRL (fixed TN-init)**                | 8.9×10⁻⁴            | **6**  | **9**  | **15**   |\n|               | TensorRL (trainable TN-init)                | 2.0×10⁻⁴            | 36     | 30     | 146      |\n|               | $\\vdots$                            | $\\vdots$        | $\\vdots$     | $\\vdots$    | $\\vdots$       |\n|               | TN-VQE                                      | 2.7×10⁻³            | 10     | 14     | 6        |\n| 10-H₂O        | **TensorRL (fixed TN-init)**                | 4.1×10⁻⁴            | **17** | **15** | **17**   |\n|               | TensorRL (trainable TN-init, χ=3)           | 6.7×10⁻⁴            | 33     | 34     | 168      |\n|               | TensorRL (trainable TN-init)                | 7.1×10⁻⁴            | 35     | 48     | 173      |\n|               | $\\vdots$                            | $\\vdots$        | $\\vdots$     | $\\vdots$    | $\\vdots$       |\n|               | TN-VQE                                      | 1.1×10⁻³            | 26     | 51     | 14       |\n| 12-LiH        | **TensorRL (trainable TN-init)**            | **1.0×10⁻²**        | 31     | 37     | 203      |\n|               | TensorRL (fixed TN-init)                    | 2.4×10⁻²            | **15** | **30** | **9**    |\n|               | $\\vdots$                            | $\\vdots$        | $\\vdots$     | $\\vdots$    | $\\vdots$       |\n|               | TN-VQE                                      | 8.6×10⁻²            | 33     | 81     | 29       |\n\nThe table show that TN-VQE consistently underperforms the RL-based approaches, both in terms of solution accuracy (error) and circuit efficiency (depth, CNOT, and ROT). Notably, for the 6-, 8-, and 12-qubit molecules, TN-VQE is unable to find the ground state with comparable accuracy to the RL-based methods, and in some cases fails to solve the problem. For the 10-qubit, TN-VQE does find the ground state, but the achieved accuracy remains significantly worse than other strategies. This demonstrate that having a high-quality initialization alone is insufficient; the RL framework provides a substantial advantage by better guiding the circuit construction and parameter optimization. Thus, our RL approach is not merely leveraging a good starting point, but also substantially improves the search strategy, enabling the discovery of circuits that are both more accurate and more efficient. We will update the **Table  1** with these results in the **Results** section. \n\n### **...How much does this choice affect the final performance? Have you explored other mapping schemes, such as generating deeper or differently structured initial circuits? Could the current results be overstated due to this highly optimized initial structure?**\n\nThank you for raising this important point. Due to time constraints and substantial code changes required for generating differently structured initial circuits, we were unable to explore alternative circuit structures in this work. Instead, we focused on varying the number of TN layers (1-3) to increase the circuit depth within the same mapping, and evaluated across 6-, 8-, and 10-qubit cases. We acknowledge this limitation and plan to investigate different TN circuit structures in future. The RL performance and circuit parameters are in the table below:\n\n| Qubit count | TN layer | Success rate | CNOT | ROT | Depth | Error     |\n|-------------|----------|--------------|------|-----|-------|-----------|\n| 6           | 1        |  0.8478       | 5    | 12  | 7     | **0.00006**   |\n|            | 2        |  0.8418       | 3    | 11  | 5     | 0.00012   |\n|            | 3        |  0.8486       | **2**    | **4**   | **4**     | 0.00024   |\n| 8           | 1        |  0.1423       | 9    | 15  | **6**     | **0.00089**   |\n|            | 2        |  0.1398       | **8**    | **11**  | 9     | 0.00133   |\n|            | 3        |  0.0000       | 6    | 14  | 7     | 0.00196   |\n| 10          | 1        |  0.1288       | 15   | 17   | 17     | **0.00041**   |\n|           | 2        |  0.1097       | **13**   | **8**   | **9**     | 0.00058   |\n|           | 3        | 0.0000       | 18   | 12  | 16    | 0.00178   |\n\nFor 6 qubits, both 2- and 3-layers TN initializations perform well, yielding similarly high success rates and competitive circuit costs; however, for 8- and 10-qubit, increasing the TN initialization depth to 3-layers actually causes the RL framework to fail (resulting in zero successful episodes), while the 2-layer initialization maintains reasonable performance. This indicates that deeper or more complex initial circuits, in this brickwork pattern, can hinder rather than overstate algorithmic performance. Thus, our results suggest that the method’s success is not simply due to an overly optimized or \"overfitted\" initial structure; rather, our chosen TN initialization offers a practical balance between expressivity and learnability. While further exploration of diverse and deeper circuit mappings is a valuable direction for future work, these findings demonstrate that our current initialization does not artificially inflate our results. We will include these additional results and analyses in the **appendix** in revised manuscript.\n\n### **...How does the search difficulty of this new problem compare to the original one?**\n\nThank you for your important question. We would like to clarify that the TN initialization provides a warm start for the RL agent by supplying good initial parameters, potentially allowing more efficient convergence by helping avoid less promising regions of the optimization landscape. However, since the underlying Hamiltonian, whether for quantum chemistry, TFIM, or Heisenberg, remains unchanged, the fundamental computational complexity of the ground state search (which is QMA-hard [[Kempe, J., Kitaev, A., & Regev, O. (2006). The complexity of the local Hamiltonian problem. SIAM Journal on Computing, 35(5), 1070-1097.]) is not altered; thus, while TN-initialization can make the process more practical, it does not change the intrinsic difficulty of the original problem. We hope this addresses the impact and intent of the TN-init method in our work. We will clarify this in the **Methods** section.\n\n### **..The mere 1% improvement for the 20-qubit system in Appendix E.6 suggests this MPS structure may struggle to represent entanglement in more complex systems, casting doubt on its generality and true scalability.**\n\nThank you for raising such a crucial question. It should be noted that the improvement (compared to the initial TN energy) depends on action space construction, the bond dimension in DMRG and number of layers in the mapping. Here to address your concern instead of considering (*RX*, *RY*, *RZ*, *CX*) as action space we add more expressive gates such as (*XX*, *YY*, *ZZ*) to the action space which enhances the improvement from 1% to 9%:\n| Problem                 | Method                                  | Error           | Improvement |\n|-------------------------|-----------------------------------------|-----------------|-------------|\n| 20-TFIM (bottleneck)    | TensorRL (fixed)                | $6.7 \\times 10^{-4}$      | 1%          |\n| **20-TFIM (action space: *XX*, *YY*, *ZZ*, *RX*, *RZ*, *RY*, *CX*)** | TensorRL (fixed)                | $\\mathbf{6.1\\times10^{-4}}$                | **9%**      |\n\nHence the bottleneck of not improving in the ground state preparation task can be addressed by a well defined action space construction. In future work we plan to explore other factors mentioned above to achieve a greater improvement. We will update the **Table** in **Appendix E.6** in the revised version of the manuscript.\n\n### **The title argues for scalability, yet the experiments are demonstrated up to 20 qubits. In the current stage, \"scalable\" is more appropriately used for systems approaching 50+ qubits, making the claim seem premature.**\n\nThank you for your feedback on our manuscript title. We appreciate your point that the original use of \"scalable\" may be premature. To more accurately reflect the current contribution we have revised the title to:\n\n*\"TensorRL-QAS: Reinforcement learning with tensor networks for enhanced quantum architecture search\"*\n\nWe believe, this new title places emphasis on the methodological advancements our approach offers to quantum architecture search.\n\nWe hope these modifications are sufficient and addresses all your concerns. Kindly let us know if you have any more comments, questions or suggestions."}, {"review_id": "WJsWH6oU1H", "reviewer": "Reviewer_e9Vh", "summary": "The paper proposes using a tensor network simulation of a parameterized quantum circuit to warm start a variational simulation of a quantum chemistry problem.  The variational simulations are performed classically using the qulacs simulator, allowing the systems to be simulated in both closed and noisy environment, on a quantum architecture of their choosing.  The authors claim that this method reduces the circuit depth required to find high quality solutions by a substantial amount (see table 1).", "questions": "Why is bond dimension 2 sufficient for these simulations?  I would have expected these simulations to need a bond dimension of $2^5$ - $2^7$  for the 8 qubit case, and more for a larger simulation.\n\nIf bond dimension 2 is sufficient for these simulations, why haven't you scaled the simulation to significantly larger system sizes (50+ qubits) when in principle an 8 qubit matrix product state implementation should take a very small amount of memory to store classically?\n\nWhat tensor network implementation are you using?\n\nHow can you guarantee that these results will continue to scale after the point where classical algorithms break down?", "limitations": "There are no apparent negative societal implications of their work.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The paper proposes a rather interesting idea, with regards to identifying useful architectures for a variational quantum circuit, but unfortunately they simply do not present enough information for it to be clear that their results are correct, or if they are correct, that they would be generalizable to a system that isn't classically simulable.  I will start with the minor issues, then lead into the bigger issues\n\nThere are a decent number of typos in the paper (which is okay, as it is very possible that English is not the first language of the authors).  Regardless of the reason, the paper needs to go through another round of copy editing.  A few examples that come to mind are for example on lines 52,53, where they say \"8 qubit\" and \"4 qubit\", when it would make more sense to say \"8 qubits\" or \"4 qubits\".  There are other typographic issues throughout the paper, but unfortunately I don't have the time to list each of them for easy correction.  I will note that in equation 2, the sum over the indices should be $i_1, i_2, ..., i_n \\in (0, 1)^{n}$ and $k \\notin (1,n)$ rather than $k \\neq (1,n)$ (note that the reviewer portal won't let me use escaped curly braces, so I use (...) to indicate a set) since this is an issue of mathematical correctness.  They also don't define $D$, which I suspect is the circuit depth on the quantum device, but it could be something else.  \n\nAssuming that their methods are correct, it is not entirely clear to me how well these methods would scale if they were only applied to a regime where fully classical workflows would be sufficient, but it could yield a slight improvement, functioning effectively as a memory-time tradeoff for systems just out of reach of classical methods currently.", "was_revised": true, "final_justification": "The work strikes me as sound, and the authors use a somewhat interesting approach.  It is not entirely clear how substantive the advantage over fully classical workflows will be in full scale simulations, but it seems to have a decent chance of giving some improvement, most likely in accuracy, when a fault tolerant quantum simulator of the appropriate scale is available.", "rebuttal": "We sincerely thank you for the valuable feedback on the manuscript, and appreciate that you agree on the interesting idea motivating our work. In the following we systematically provide the response to your comments.\n\n### **There are a decent number of typos in the paper (which is okay, as it is very possible that English is not the first language of the authors). Regardless of the reason, the paper needs to go through another round of copy editing...**\n\nWe acknowledge the presence of typos and missing definitions, we will perform a thorough proofreading of the manuscript to correct all exposition issues.\n\n### **Why is bond dimension 2 sufficient for these simulations? I would have expected these simulations to need a bond dimension of $2^5$-$2^7$ for the 8 qubit case, and more for a larger simulation**\n\nYou have raised a valid concern regarding the use of a low bond dimension, and we now clarify that this choice is in fact intentional. For the relatively small quantum chemistry problems considered, a larger bond dimension would be sufficient to produce solutions that are already within chemical accuracy of the ground state energy. In this sense, the low bond dimension is not a limitation but rather an intentional choice: using a larger bond dimension would allow us to solve the problem classically, without the need of any additional RL-based quantum computing resource.\n\nWe emphasize that the current study is a proof-of-concept intended to validate the proposed combination of TN and RL-QAS, and we start testing it on small systems due to the heavy computational requirements needed to train RL agents (see the next discussion). Applying the same methodology on a larger number of qubits would probably require larger bond dimensions in the MPS/DMRG, which is consistent with your expectation.\n\n*We also clarify a technical point:* for an MPS (with open boundary conditions), the maximum required bond dimension to represent an arbitrary n-qubit state exactly is $2^{⌊n/2]}$. For n=8, this means that a bond dimension of $2^4$ is the theoretical maximum needed to express any arbitrary state exactly, not $2^5$ or $2^7$ as suggested. We have clarified this in the revised manuscript to avoid confusion.\n\n### **If bond dimension 2 is sufficient for these simulations, why haven't you scaled the simulation to significantly larger system sizes (50+ qubits) when in principle an 8 qubit matrix product state implementation should take a very small amount of memory to store classically?**\n\nThank you for the interesting question. We want to clarify that the main computational bottleneck in our simulations does not lie in the MPS/DMRG simulations (it is inexpensive as long as the bond dimension is not too big), but in the RL training phase. Each training episode requires simulating and optimizing a parameterized quantum circuit, and computing its expectation value multiple times equal to the number of (episodes X steps). Already for ~12 qubits, this becomes computationally intensive due to the number of episodes and steps required for convergence/solving the task. This currently limits our ability to explore system sizes beyond ~20 qubits, as already at these sizes training the RL agent can take approximately 48 hours, which is the maximum runtime available on the University computing cluster which is currently available to us. Future work will address this by further alleviating the computational requirements of the RL training process, and investigating more performant quantum circuit simulators.\n\nSimulating quantum circuits is computationally very hard in general, which is indeed the reason why we propose to combine a classical warm-start with an RL agent that completes the task by accessing quantum resources directly. But since actually accessing quantum hardware is hard because of limited access by providers, we are bound to do simulations of quantum circuits. Finally, to the best of our knowledge, we remark that our work reports the largest RL-based QAS simulation to date with 20 qubits. \n\n### **What tensor network implementation are you using?**\n\nWe use the *Quimb Python library* for all tensor network computations and MPS manipulations, including DMRG simulations and mapping to quantum circuits. \n\n### **How can you guarantee that these results will continue to scale after the point where classical algorithms break down?**\n\nWe acknowledge this is a central open question for all hybrid classical-quantum algorithms, including ours. Like most near-term quantum algorithms, we cannot provide theoretical guarantees that our TensorRL-QAS approach will outperform classical methods at much larger scales. However, we believe our strategy of maximizing classical pre-processing with tensor networks not only will help reduce the quantum resources required, which is especially important for the near- and mid-term regime, but also very much aligns with the current expectation that useful quantum advantage will be achieved with a tight combination of quantum and classical high-performance computing. Overall, we are confident that TensorRL-QAS provides a necessary starting point for future investigations on larger sizes, where a quantum advantage may manifest. We have added explicit discussion of these considerations and limitations in the revised manuscript. \n\nWe hope these addresses your concerns and questions and further helps you to re-evaluate our paper. Kindly let us know if you have any further comments or/and questions."}]

Author rebuttal:

Meta-review: This paper studied the problem of ground state preparation using quantum architecture search framework. It utilizes tensor networks especially the DMRG method to prepare an initial ground state, and then applies reinforcement learning to further improve the solution. This framework reduces function evaluations by up to 100-fold, accelerates training episodes by up to 98%, and achieves up to 50% success probability for 10-qubit systems.

Reviewers found this paper with the following strengths:
- The idea of combining DMRG/MPS tensor network initialization with RL for QAS is novel.
- The results of achieving up to 10× CNOT reduction and 100× computation time reduction are competitive.
- The framework has robust performance in noiseless and noisy settings and the outperforms RL and non-RL baselines.

Nevertheless, in the submitted version the reviewers also raised concerns, including scalability, experimental scope, impact of bond dimension, ablations & baselines, etc. The authors addressed these concerns adequately during the rebuttal, and all reviewers reached consensus in accepting this paper. The AC followed this decision accordingly.

Nevertheless, for the final version of the paper, the authors should make the following changes:
- Implement all the promised changes mentioned during rebuttal.
- Include all tables on new experiments conducted during rebuttal.
- When the AC checked the paper, two further points are observed and would be good if the authors can improve: 1) The main body of the paper keeps discussing RL-based algorithms, but didn't introduce much on which RL algorithm was used. It was only vaguely mentioned in Fig. 1 that DDQN was applied. For the NeurIPS audience, it would be helpful to introduce in the main body which RL algorithm was used with which parameter settings, and maybe at least in appendices a comparison between different RL algorithms and why DDQN was the most appropriate choice. 2) There's an important paper on quantum architecture search, the QuantumNAS paper https://ieeexplore.ieee.org/abstract/document/9773233, that was not cited. The authors should discuss about this one as well as some of its follow-up works in quantum architecture literature.
Decision: Accept (poster)