Reinforced Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Authors: Manwen Liao
Yan Zhu
Weitian Zhang
Yuxiang Yang
Venue: ICML 2025 poster (2025)
Keywords: AI for Science
Quantum Property Estimation
Quantum State Characterization
Reinforcement Learning
Primary area: applications->chemistry_physics_and_earth_sciences

Abstract: Characterizing quantum states is essential for advancing many quantum technologies. Recently, deep neural networks have been applied to learn quantum states by generating compressed implicit representations. Despite their success in predicting properties of the states, these representations remain a black box, lacking insights into strategies for experimental reconstruction. In this work, we aim to open this black box by developing explicit representations through generating surrogate state preparation circuits for property estimation. We design a reinforcement learning agent equipped with a Transformer-based architecture and a local fidelity reward function. Relying solely on measurement data from a few neighboring qubits, our agent accurately recovers properties of target states. We also theoretically analyze the global fidelity the agent can achieve when it learns a good local approximation. Extensive experiments demonstrate the effectiveness of our framework in learning various states of up to 100 qubits, including those generated by shallow Instantaneous Quantum Polynomial circuits, evolved by Ising Hamiltonians, and many-body ground states. Furthermore, the learned circuit representations can be applied to Hamiltonian learning as a downstream task utilizing a simple linear model.

Reinforced Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Manwen Liao 1
Yan Zhu
Weitian Zhang² Yuxiang Yang'
Abstract
technologies. There two main approaches to tackling this task: classical methods and quantum methods. Classical methods, such as quantum state tomography (Tóth et al., 2010; Gross et al., 2010; Cramer et al., 2010; Lanyon et al., 2017; Cotler Wilezek, 2020), reconstructs the quantum state by measuring an informationally complete set ob- servables. These methods require exponentially increasing sample complexity in measurements as the size of the quan- tum system grows, making them impractical for systems with many qubits and thereby limiting their applicability for practical use. Quantum methods, represented by varia- tional quantum algorithms (Cerezo et al., 2021), utilize the power of quantum circuits to learn quantum states. These methods (Peruzzo al., 2014; Farhi et al., 2014; Du et al., 2022; Wu al., 2023a) typically optimize parameterized quantum circuit approach the target quantum state. Nev- ertheless, due to the necessity of calculating gradients with respect to circuit parameters, where the loss landscape is highly flat, these methods often struggle with issues such as barren plateaus (McClean et al., 2018; Cerezo et al., 2021) and local minima (Anschuetz. & Kiani, 2022; Huang et al., 2024), consequently affecting their performance in learning large-scale quantum systems.
Characterizing quantum states is essential for ad- vancing many quantum technologies. Recently, deep neural networks have been applied to learn quantum states by generating compressed implicit representations. Despite their success in predict- ing properties of the states, these representations remain a black box, lacking insights into strategies for experimental reconstruction. In this work, we aim open this black box by developing explicit representations through generating surrogate state preparation circuits for property estimation. We design a reinforcement learning agent equipped with a Transformer-based architecture and a lo- cal fidelity reward function. Relying solely on measurement data from few neighboring qubits, our agent accurately recovers properties of target states. We also theoretically analyze the global fidelity the agent can achieve when it learns a good local approximation. Extensive experiments demonstrate the effectiveness of our framework in learning various states of up to 100 qubits, in- cluding those generated by shallow Instantancous Quantum Polynomial circuits, evolved by Ising Hamiltonians, and many-body ground states. Fur- thermore, the learned circuit representations be applied to Hamiltonian learning a down- stream task utilizing simple linear model.
In practice, the primary interest often lies in specific prop- erties of quantum states rather than their full state repre- sentations. As a result, full quantum state tomography is not always necessary, especially when the goal is to pre- dict certain physical observables. Motivated by this, re- cent approaches integrate machine learning techniques to construct "shadow" representations original states, enabling efficient quantum system charac- terization. These methods have shown success in quantum state learning (Carleo & Troyer, 2017; Sharir et al., 2020; Zhu et al., 2022; Zhang Di Ventra, 2023; Tang et al., 2024a; Chen & Heyl, 2024; Du et al., 2023; Qian al., 2024), quantum process learning (Huang al., 2023; Torlai et al., 2023; Zhu al., 2023), quantum property estima- tion (Zhang & Di Ventra, 2023; Wu et al., 2024; Lewis et al., 2024; Tang et al., 2024a), quantum state classification (Tang et al., 2024b), quantum sensing (Xiao et al., 2022; Zhou et al., 2024) and quantum verification (Wu et al., 2023b; Qian et al., 2024). By leveraging neural networks to ef- ficiently represent quantum states, these methods signifi-
surrogate to the
1. Introduction
Quantum state characterization is a critical task in quan- tum information, underpinning the development of quantum computing, quantum communication, and quantum sensing
"Equal contribution 'QICI Quantum Information Computa- tion Initiative, Department of Computer Science, School of Com- puting and Data Science, The University of Hong Kong. Pokfulam Road, Hong Kong, China MMoE Key Lab of Artificial Intelligence, Al Institute, Shanghai Jiao Tong University, Shanghai, China. Cor- respondence to: Yuxiang Yang <yuxiang @cw..ku..k.........
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Table 1. Summaar of quantum state characterization methods. #Observables: The number of observables utilized for characterizing the target quantum states. Experimental Reconstructability: The ability to construct a quantum circuit to prepare the surrogate state from its measurement data. Downstream Applicability: The capability to perform downstream tasks, such as Hamiltonian learning. based on the classical representation. Scalablity: The ability to extend the learning scheme to large-scale quantum systems (e.g.. N > 20 qubits).
Experimental Reconstructability
Downstream Applicability
Mitigale
Methods
#Observables
Sculability
Barren Plateaus
Peruzzo et al. (2014)
X
Quantum
Farhi al. (2014)
Tóth et al. (2010); Cotler & Wilczek (2020)
(
X
x
X
Carleo & Troyer (2017); Chen & Heyl (2024)
O
X
/
Classical
Zhu et al. (2022)
((N)
X
/
Ours
(NN)
cantly reduce the measurement overhead. Approaches such as generative neural networks (GQNQ) (Zhu et al., 2022) and LLM4QPF (Tang et al., 2024a) approximate quantum states using fewer measurements by exploiting underlying patterns and correlations shared within a family of states. By learning compact and expressive representations, these techniques offer scalable solutions for quantum state char- acterization, making them particularly valuable for learning large, complex quantum systems where traditional meth- ods become impractical. However, these representations are often implicit. While they capture essential features and properties of the quantum state, they do not support direct reconstruction of the state from the representation itself. This limitation poses challenges in scenarios where an explicit reconstruction of the quantum state is required, e.g., quantum phase estimation (Kitaev, 1995) and quantum simulation (Georgescu et al., 2014).
we design novel reward function based on local fidelity, and provide a theoretical analysis on the effectiveness of reconstructing global properties given local information of the states. The contributions are:
(1) We develop a novel type of representations for quantum states, termed the explicit circuit representations. Unlike conventional implicit state representations in GQNQ (Zhu et al., 2022) and Neural Quantum State (NQS) (Carleo & Troyer, 2017: Sharir et al., 2020; Zhang & Di Ventra, 2023; Chen & Heyl, 2024), our circuit representations can be directly utilized to experimentally reconstruct the target states locally, which allows for computing the properties of interest via measuring the output states. Morcover, they possess the advantage of implicit representations that can be applied to downstream tasks.
(2) We design a reinforcement learning-basee framework named QCrep to learn explicit circuit representations for specific families of quantum states using only measurement data from a small number of neighboring sites. The circuits learned by QCrep can reproduce the target states with high global fidelity, utilizing (NV) observables with respect to the system size N. Benefiting from our novel design, QCrep eliminates the need for gradient- based optimization of cir- cuit parameters and mitigates the barren plateaus problem, enabling scalability to larger systems of up to 100 qubits.
In this work, we propose a novel type of representations, the explicit circuit representations, formulated by sequence of tokens, to characterize quantum states and design deep reinforcement learning-based framework named QCrep to learn such representations that can experimentally recon- struct a surrogate for the target states for property estimation. A comparison of different methods for state characterization is shown in Table 1. Two main challenges for learning the circuit representation are the high measurement overhead and the barren plateaus problem. The high measurement overhead roots from the fact that exponential number of measurements is required to fully characterize an unknown quantum state. However, this can be surpassed when tar- geting at learning an approximation for accurate property estimation. We only use local measurements on a few neigh- boring sites of the quantum states to construct local state representation for the target state, Additionally, we propose novel Transformer-based (Vaswani et al., 2017) measure- ment feature aggregation block to recover global features of target states from local measurement data. To mitigate the problem of barren plateaus and local minima, we involve deep reinforcement learning that does not require computing gradients with respect to the circuit parameters. Besides,
(3) We experimentally demonstrate the effectiveness of our framework by learning four different families of target states and applying it to Hamiltonian learning (Wiebe et al., 2014; Wang et al., 2017) as downstream task. Our framework achieves superior performance in learning states generated by Instantaneous Quantum Polynomial (IQP) circuits (Brem- ner et al., 2010), states evolved by Ising Hamiltonians, and ground states of many-body quantum systems. For the downstream task, numerical experiments reveal that the unknown parameters of Hamiltonians can be accurately learned from local measurement data of their corresponding ground states, leveraging only a linear model applied to the circuit representations. This further highlights the versatility and effectiveness of our framework.
2

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
2. Learning Explicit Circuit Representations
ment M; (May) is a positive operator-valued mea- sure (POVM) acting on two neighboring qubits (i, i + 1) of Ps. satisfying Σه Ma = Specifically, we select the measurement operators Mi as the tensor product of two single-qubit Pauli operators, i.e., Ma E (X Z)02 We measure each pair of neighboring qubits using all such oper- ators in fixed order, taking the expectation values of the measurements to obtain the measurement output m, E IR where K = 9. We repeat this process for all qubit pairs and record the measurement data as m E R
2.1. Task Definition
We define the task of learning explicit circuit representations for quantum states as characterizing family of unknown quantum states S { by constructing quantum cir- cuits 4- (U,)= ) that can prepare these states with high local fidelity, so that the reconstructed states can be directly measured to predict quantum properties of interests. We assume that the states can only be accessed in a black-box manner, meaning one can measure the states using measure- ment operators M but remains agnostic to the underlying circuits used for their preparation. Additionally, we assume that the measurement operators can only act on neighbor- ing sites of the quantum states, a setup we refer to as local measurements. This measurement configuration has been widely adopted in prior works on quantum state character- ization (Lanyon et al., 2017; Friis et al., 2018; Zhu et al., 2022; Kurmapu et al., 2023; Guo Yang, 2024; Wu et al., 2024) due to its feasibility for experimental realization. For this learning task, we do not put explicit constraints on the global fidelity between the reconstructed states and the target states, but focus on maximizing the local fidelity.
Importantly, the measurement operators are discarded when we input the measurement data into the agent. The cor- respondence between the operators and their expectation values is expected to be reconstructed during training. Note that although quantum states and measurement operators are represented by complex- expectation values are real and range from -1 to 1, since the eigenvalues of Pauli operators are either 1 or This property. along with the removal of measurement operators from the neural network's 's input, exempts the framework from the overhead of processing complex values.
numbers, the mcasurement
2.3. QCrep Framework
Explicit Circuit Representations. Let P. be an N-qubit quantum state that we want to characterize. The recon- structed state as = U. 0) (o]@NUt possesses high aver- age local fidelity with the target state P.. thus can serve as a surrogate for predicting quantum properties of inter- est. Ug can be expressed
To construct circuit representations for reproducing fam- ily of quantum states, we design reinforcement learning- based framework, QCrep. This framework relies exclusively on local measurements and avoids performing gradient de- scent on circuit parameters, effectively mitigating the barren plateau problem. The overall pipeline is shown in Figure I, A deep reinforcement learning agent utilizing neural net- work policy is employed to construct the circuit represen- tations for family of unknown quantum states S. The environment in which the agent interacts and learns is de- fined as the quantum system. This environment is initialized with the quantum state to be learned, p!0) -a E S, and is responsible for applying gates to the state as the agent itera- tively learns to reconstruct the state. The observations are the local measurement values m, obtained in Section 2.2. We define the actions that the agent can take at step / as applying a layer of quantum gates to the state. The reward function is the local fidelity reward defined in Equation 3.
product of unitary gates, i.c., U. = II, (6..)..)). where U..: represents the quantum gates applied at time step 1, and 6a.t denotes the corre- sponding parameter(s) for those gates. The explicit circuit representation of Ps is a sequence of (ua,t: &e,)).. where Us is the classical description of the gate type of U..-- The correlations between the gate types and actual gates are encoded using look-up table.
Overview. To learn the circuit representations, we first perform local measurements on the target states, which is introduced in Section 2.2. After that, we design a rein- forcement learning-basee framework, QCrep, to decode the measurement data into quantum circuits, and keep the classi- cal descriptions of the circuits as the circuit representations. This is described in Section 2.3, wherein a measurement feature aggregation block is proposed to process the local measurements, and a local fidelity reward function is de- signed to ensure learnability. Background information on quantum computation introduced in Appendix B.
Instead of directly learning U.. the agent is trained to con- struct V, I Vo..( = U.. which evolves Ps to- wards [0) (0) @. where Va, represents layer of quantum gates chosen at step f, and Φ.,, is the corresponding gate parameter. This approach enables the learning of family of quantum states, as directly learning U, requires fixed input state 10) (0) @o which limits it to learning a single state. In contrast, by evolving towards 10) (0) S.. any state can be set as the input, facilitating the learning of a family of states. The target U. can then be obtained via taking the inverse
2.2. Measurement Setup
We consider a set of measurements M- = (M,) N-2 termed local measurements, performed on neighboring sites of the unknown N--uubi quantum states Ps. Each measure-
of Vs. ie.. U, = Vi = Vi Vo,( Note that during
3

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Observation
Unknown Quantum States Family S
P
Local Measurements
Training w/ Local Fidelity Reward
Environment
Observation
Agent
Agent
Positional Encoding
Multi-head Self Attention
PPO Update
0) (0)eN
(0)
P
Linear & Layer Norm
Local Fidelity Reward
Action Vi(ó.)
Sample
[c-iou a X,Vs, ..... leos Zen (Yu]
Feature A Aggregation
Sample
MLP
(t) Ps
Vi(
+
Ps
(1+1)
Figure 1. QCrep framework. Given an initial state pa sampled from an unknown quantum states family S, the agent iteratively applies quantum gates Ve( evolve state towards (0)((0) (0 The policy parameterized by neural network, which includes an Attention-baas based measurement feature aggregation block followed by Multilayer Perceptron (MLP). The agent is trained using the PPO algorithm with a local fidelity reward.
the circuit construction procedure, only the local fidelity between an arbitrary state and [0] 8N needs to be evaluated, which can be obtained via Pauli Z measurements. This empowers the practical implementation our framework.
in the experiments. Note that this T can be flexibly ad- justed to control the accuracy of the reconstructed states or to meet hardware requirements when implemented on real quantum computers. The measurement complexity scales linearly with T, because for each I, only constant number of measurements is performed if the system size is fixed. The policy Ta is updated using Proximal Policy Optimization (PPO) algorithm (Schulman et al., 2017).
The entire process of learning the circuit representation for P. consists of several iterative steps. At each step t, the state Ps measured using local measurement operators and the agent takes the expectation values (*) as observations from the environment. Then, the agent selects the action (o,.)) according to its policy Ta. which parameter- ized by trainable Gaussian distribution generated from neural network composed of feature aggregation block followed by Multilayer Perceptron (MLP). The action Ve.(().))) 2x (0e.,))) is column of single-qubit or two-qubit gates acting in parallel to every qubit i, where Vo...,(0..,.)) exp (-icb...::)) are generated from the lin- ear combination of the single-qubit and two-qubit Pauli operators, namely, G span ((X, Y, 2} U (X, Y, 82)) To further reduce the search space, we adopt task-aware fashion select subset of gates the action space, which will be detailed in Section 3. After that, the environment up- dates the quantum state as ps (++1) (v,,()..))
Or+ arg max (m, (od))~aaa (J m, V(Ø), one)]
and
Ta (V(())) Tax
Ja, m. V(0), min
Tax (V())
Ta (V (m)
clip,
AT
Tay, (V())
where A A0 the estimated advantage function associated with reward r, and 8 measures the gap between the new and old policies. Finally, we keep the sequence of classical de- scriptions of the quantum gates (u 0x,0) the circuit representation of P.-
v (o..)) and the agent receives a reward (1) defined in Equation 6. We repeat the above procedure until the average local fi- delity L |o)<0|$A &N), defined in Equation 3, exceeds threshold of or until the number of iterative steps reaches a predefined maximum of T. We set e 0.001
A Attention-based Measurement Feature Aggregation Block. We construct a novel feature aggregation block to map the quantum measurement data m to a compact vector representation p. There are two main features for this block: (1) Transformer (Vaswani et al., 2017) module is proposed

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
to capture the entanglement property of the quantum states from local measurement data. Due to the entangled nature of quantum states, non-local correlations exist among qubits, leading to long-range dependencies between measurement values. Therefore, we utilize self-attention to model the dependencies between different qubits. (2) An aggrega- tion layer, implemented as global average pooling along the sequence axis (the second axis), is introduced to globally model the state. This enables transferability across quantum systems of varying sizes, allowing the framework to perform zero-shot transfer learning. Ablation studies demonstrating the effectiveness of this component compared to vanilla MLP are presented in Appendix I.
Proposition 2.1. If the agent learss a policy that constructs N-qubit quantum state with average n-local fidelity L (ps (0) (0)@N) 2 1 --- 6, then the global fidlity satis-
fies F(ps (T) (0)(N)))) 2 21= 11--(3 (N 12 1)E.
This presents lower bound for the global fidelity when the agent learns good approximation of the local fidelity. The bound is derived without assumptions on the entanglement of the target states. However, one expects that for low- entangled states, the gap between local and global fidelity becomes much higher than - (N - n + 1)e. This phe- nomena exists commonly in quantum many-body systems. We will show in experiments that in addition to predicting properties, the learned states can even achieve high global fidelity with the target states.
Local Fidelity-based Reward Function. Training based on global fidelity is prone to be trapped by barren plateaus (Mc- Clean et al., 2018; Cerezo et al., 2021; Bittel & Kliesch, 2021; Larocca et al., 2025). To mitigate this effect, we propose novel reward function based on average n-local fidelity, inspired by the use of local cost functions to miti- gate barren plateaus (Cerezo et al., 2021; Caro et al., 2023). Given two N-qubit quantum states p and a, the average n-local (1 S n S N) fidelity is defined as
3. Experiments
In this section, we apply our framework to learn cireuit rep- resentations for four different families of states the states prepared by Instantaneous Quantum Polynomial (IQP) cir- cuits, states evolved by Ising Hamiltonians, and two types of quantum many-body ground states. In addition, we use Hamiltonian learning as an example to showcase the inter- pretability of circuit representations learned by our model.
I (p,o) N n
1
F(pestny
where F is the (global) fidelity between the reduced density matrices Prsstn and ozstn of the original states on qubits i + ,,..., ; + n - 1}. This reward is derived exclu- sively from local measurements. In our scenario, we set @n and the average n-local fidelity, denoted can be estimated by measuring p using local operators {O(") JN"", where
Our framework is compared with Transformer Quantum State (TQS) (Zhang & Di Ventra, 2023), Variational Quan- tum Eigensolver (VQE) (Peruzzo al., 2014), Quantum Approximate Optimization Algorithm (QAOA) (Farhi et al., 2014) and Quantum Architecture Search (QAS) (Du et al., 2022). The metrics for evaluation are second-order Rényi entropy (Rényí, 1961), two-point correlations (Fetter & Walecka, 2003), spin-Z values (Atkins de Paula, 2010), and square root global fidelity. The definitions of them are introduced in Appendix D. For the former three metrics, we compute the Root Mean Squared Error (RMSE) between the ground-truth values measured form the target states, and the actual values obtained form the learned representations / output states. For fair comparisons, the training objective for all methods ods and local fidelity reward for ours. All properties are predicted once trained without fine-tuning, The simulation details for the experiments are presented in Appendix E.
Juitn 10) (0)@n
as I (m) (p5 (t)
, 10) (0) @NN, (o)e))
00Th 0) (0) u0 @ INAii
(4)
which applies a projector (0)(0)@n @n to qubits (i...., i n
1). and identity to the remaining qubits. The overall operator associated with the average local fidelity is defined as
O (+)
Σ m
(5)
N -
sieeittyglobar fidelity for the other meth-
We can compute average local fidelity between the state at step and the target state as (p("), [0)
Tr(O (f)) ( The reward for the agent is defined as
(0) = -1 -1
(0) @N)
(6)
3.1. Learning Quantum States Generated by Instantaneous Quantum Polynomial Circuits
An additional -1 term added into the reward to encourage generating cireuits with lower depth. In our experiments, we fix n = 1 for efficiency. Results on circuit learning with larger n is presented in Appendix H.I. To bound the accu- racy of the circuit representation trained with this reward function, we present the following proposition (the proof is given in Appendix C):
IQP circuits are frequently used to benchmark the classical simulatability of quantum circuits (Bremner et al., 2010). While general IQP circuits are classically intractable to simulate, in this experiment, we focus on a specific family of states generated from shallow circuits to demonstrate the capabilities of our framework. The output states generated
5

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
by IQP circuits are
Table 2. Evaluation results of learning states generated by 10-quhit IQP circuits.
N-1 M+) HiZ[ajs H 0m
N-1
Method
Rényi Entropy 4 Correlations L
Spin-Z.4
Global Fidelity f
0
TQS
0.5765
0.2906
0.2309
0.6894+-0.298 2946
VQE
0.1665
0,1100
0.1539
0,9174=0.100
where Z [[a]s are single- or two-qubit gates that can be diag- onalized in the computational basis, e.g., Z. CZ and (a). In our setting, Z[o]i contains one column of CZZ gates act- ing on every two adjacent qubits, followed by one column of single-qubit gates randomly sampled from R2 (a). where E [- ; x 2 for each qubit. We consider a quantum system with a size of N = 50. We generate 100 different circuits and record the output states as our training set. The quantum eircuits are discarded once the states are generated. We train our framework reconstruct the circuits that pre- pare the target states in the training set. The action space for generating circuit representations is {H, C'Z, Rs (o)). where Φ is determined by the agent. During training. we set the maximum number of íterative steps to T = 100. In each step, one gate is applied to one qubit or two nearest neigh- bor qubits. After training, we generate another 10 different states for evaluation. We use global fidelity and local fidelity as metries between the reconstructed states and the target states to test the learned circuit representations. Figure 2(a) and (b) show the scaling of global and local fidelity at step t. For 4-qubit states, the local fidelity increases concurrently with global fidelity. However, for 50-qubit states, while the local fidelity monotonically increases, the global fidelity remains stable and sharply rises at the end of the period. The trajectory of the evolution of global fidelity shows that in a large region corresponding to iterative steps 0 to 70, the global fidelity is almost O. This indicates that the vari- ance of the loss function over the whole parametric space is small, highlighting the existence of barren plateaus. We
QADA
0,2538
0.1026
0.1429
0.8336=:01617
QAS
0.3977
0.0895
0.2401
0,4694:0.1500
Ous
3.0737c-07
1.0902e-08
0.0441
0.9851=000208
3.2. Learning Quantum States Evolved by Transverse Field Ising Hamiltonians
Next, we consider learning the circuit representations for family of states evolved by transverse field Ising Hamil- tonians, where the exact parameters of the Hamiltonians and evolution time are agnostic to the framework. Starting with product state [0) 10)@8, the state evolved by an Ising Hamiltonian for time r. The target states after the evolution are defined as
18)) = e iHaice 7 (0) 8.N
(8)
where Hising = J'' 2; t 9 Ei-0 X, is the trans- verse field Ising Hamiltonian, t is the evolution time. In our experiment, we set N = 50, and r E [0.1, We sample 10 different gs and 10 rs uniformly from the range with stride 0.1 to construct the training set of size 100. For training, we set the maximum number of iterative steps to T = 100, each corresponds to applying one gate to each qubit or every two nearest neigh- bor qubits. The quantum gates composing the action space are (cxp(-io.X), exp(-iø @ Z)). To exhibit the results, we average the performance on different parameters g for each evolution time r. Figure 3(a) shows that the learned cir- cuit can successfully recover the target quantum states with high fidelity. Additionally, we evaluate the circuit depth and compare it to the first-order Trotter decomposition (Suzuki, 1985), which is considered one of the most straightforward methods for simulating the dynamics of quantum systems. As shown in Figure 3, our framework can construct circuits shallower than those generated by the Trotter decomposition in general. This indicates that our framework can serve as an optimization technique for traditional quantum simula- tion technologies. Notably, our framework does not require prior knowledge on the Hamiltonian parameters, offering greater flexibility compared to the Trotter decomposition method when simulating the dynamics of quantum systems. Figure 3(c) shows the zero-shot transfer performance of ap- plying the framework trained on 50-qubit systems to other quantum N N-qubit systems with N E {10, 30, 70, 100). The output states remain high fidelity with the target states of unseen sizes, demonstrating the success of our measurement feature aggregation block. In addition, we compare the per- formance of our framework with other methods for quantum
-1, 9 E 2-20 -1.0]
(a)
(b)
0..
Edellt
0.6
0.4-
0.4
Global
022
Local
02-
0
20
40
80
80
100
Iterative steps
Iterative steps
Figure 2. Learning quantum states generated by IQP circuits. (a) States generated by 4-qubit IQP circuits. (b) States generated by 50-qubit IQP circuits.
also compare the performance of our framework with other state characterization methods for quantum systems of size N = 10. The results in Table 2 show that our framework outperforms all others across all metrics.
6

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Global
Local
Ours
Trotter
Global
Local
(b)
(c) 1.0
1.00
0.98
Circuit depth
1.5
0.8
Fidelity
0.96
Fideiity
0.6
1.0
0.4
0.92
0.2-
0.90
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
0.0
100
Evolution time
Evolution time
Number of qubits
Figure 3. Learning 50-qubit quantum states evolved by Ising Hamiltonians. (a) Scaling of global and local fidelity w.r.t the evolution time. (b) Comparison of the circuit depths for simulating the dynamics between framework and the Trotter decomposition method. (c) Zero-shot transfer performance on quantum systems of various sizes. Our framework is trained on the 50-qubit system.
systems of size N 4. Table 3 illustrates the results of predicting different properties. Our framework outperforms other methods on all metrics.
ance learning quantum states, in which relaxed metric encourages exploration and increases the chance of finding the optimal result. Figure 4(c) shows the zero-shot transfer performance the framework trained on the 50-qubit sys- tem when applied system sizes of {10, 30, 70, 100). The comparison results between different methods for learning 10-qubit Hamiltonian ground states are presented in Table 4.
Table Evaluation results of learning 10-qubit states evolved by Ising Hamiltonians, where the evolution time E [0.1, 1).
Method
Réayi Entropy
Correlations
Spin-Z
Global Fidelity
TQS
0.1727
0.1037
0.0944
0.852420.0957
(a)
Global
Local
(b)
Global
Local
VQF
0.5824
0.3044
0.3619
0.2795+0.2359
1.00
QAOA
0.0324
0,0382
0.0513
0.9637-00.
QAS
0.3729
0.4349
0,4806
0.5215=0.2153
Ours
0.0108
0.0227
0.0231
0.9979=00.0012
Fidelity
0.96
Fideilyy
0.6
0.94
0.4
3.3. Learning Many-body Ground States
0.92
0.-
Our third experiment learning the circuit representations for family of many-body ground states. We consider two families of ground states separately, the transverse-fied Ising ground states, and the anisotropic Heisenberg XXZ. ground states.
0.90
-2.0
-19
-16
-1.
0.0
200
qubits
Figure 4. Learning 50-qubi, ground states of transverse- field Ising model. (a) Scaling global and local fidelity w.r.t the Ising parameters. (b) Zero-shot transfer performance on learning Ising ground states of various sizes (trained on the 50-qubit system).
Learning Transverse-field Ising Ground States. In this experiment, we consider the same Ising Hamiltonians as in Section 3.2, but with the goal of learning the ground states rather than time-evolved states. The configurations are N 50, formly sample 20 different parameters and compute the corresponding ground states, storing them into the training set. Then we use the QCrep agent to learn the circuits to prepare these ground states. The action space the same as described in Section 3.2. We the maximum number of iterative steps T 200 during training. Figure 4(a) shows the scaling of global and local fidelity with the pa- rameters The local fidelity almost remains stable but the global fidelity slightly drops with the increment of g. This indicates that in bigh-dimensional space, global fidelity is more sensitive to differences between states compared to local fidelity. Thus, global fidelity may not be a good guid-
Table 4. Evaluation results Ising systems.
of learning ground states 10-qubit
-1 and g [-2.0, -155 We uni-
Method
Rényi Entropy 1 Correlations L
Spin-Z J
Global Fidelity 1
TQS
0.1187
0.0958
0.0306
0.9537=0.0724
VQE
0.6442
0.1475
0.4773+40.0087
QAOA
0.0368
0.1009
0.0229
0.9614:000181
QAS
0.2260
0.1806
0.1706
0,8032=0.0450
Ours
0.0989
0.0947
0.0309
0.9691=000003
Learning Anisotropic Heisenberg XXZ Ground States. Here, we learn the circuit representations of ground states of a family 1-D Heisenberg XXZ Hamiltonians. The Hamil- tonian is H = NN J,X,X1441 + J,YYY++ + J J7, Throughout the experiment, we set Jj Jy
7

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
and J e [-3.0, -2.0). To construct the training set, we uniformly sample 10 different J and generate the ground state of system size N 10. The action space of the agent is {exp(-ioX @ X), exp(-iøY @ Y), exp( - @ Z)}. We set the maximum iterative steps to T = 100. The seal- ing of global and local fidelity with parameter J is shown in Figure S(a). Besides, we evaluate the trained frame- work on out-of-distribution data. We generate 9 different ground states corresponding to J E [-1.9, -..1,, and use the trained framework to generate circuit representations to reproduce these states. Results in Figure 5(b) show that our framework can successfully be generalized to prepare unseen states within the same state family. The comparison
Heisenberg XXZ model. Specifically, we first use QCrep to learn the circuit representations (vl (b){= for ground states corresponding to Hamiltonians with unknown param- eters (g in Ising model and J in Heisenberg model). Next, we concatenate the representations into vectors and pad Os at the end to ensure the same length. Finally, we employ linear regression to establish the relationship between circuit representations and Hamiltonian parameters using a small training set, and we utilize the learned framework to predict the relationship on the test set. Experimental results in Fig- ure 6 show that, given the circuit representation of a ground state associated with a Hamiltonian with unknown parame- ters, these unknown parameters can be accurately predicted using only linear regression. Meanwhile, for comparison,
(a)
1.00
100
(a)
Actual
Ours
(b)
QAOA
0.38
0.98
Hamiltomian parametes
0.96
0.96
-1.4
-22
0.94
0.94
-24
Globel
0.92
Local
-266-
-1.8-
0.90
-3.0
-2.6
0.90
Index 0
Index
Figure 55 Learning 10-qubit Heisenberg ground states. (a) Scaling of global and local fidelity w.r.t. the Hamiltonian parameters. (b) Out-of-distribution generalization.
Figure 6. The test set performance different methods on learning Hamiltonian parameters for 10-qubit (a) Ising and (b) Heisenberg XXZ quantum systems. The x-axis represents the parameter in- dices, and the y-axis shows the corresponding parameter values.
with other methods on 10-qubit system is shown in Table Our framework can accurately recover the three properties of the target states and achieves the highest performance.
we use the circuit parameters learned from VQE and QAOA to perform Hamiltonian learning. However, the linear model fails to establish relationship between the Hamiltonian and circuit parameters. We attribute this outcome to the QCrep learning pipeline, which effectively encodes information about the underlying Hamiltonian into the circuit parame- ters. This not achievable with VQE or QAOA, they rely gradient-basse based optimization of the circuit parameters, which perturbs the parameters and hinders the preservation of Hamiltonian information.
Table Evaluation results of learning ground states of 10-qubit Heisenberg XXZ systems.
Method Rényi Entropy
Correlations
Spin-Z
Global Fidelity
TQS
0.7071
0.0017
0.0159
0.6288:20.1200
VQE
0.0042
0.0816
0.7840
0.0065:000110
QAOA
0.0038
0.0693
0.0000
0.5970=000085
QAS
0.3379
0.1234
0,4962
0.7613=0.0609
Ours
0.0000
0.0000
0.0000
0.9550=20.0229
3.5. Discussions on Practical Implementation
3.4. Downstream Application: Hamiltonian Learning
Impact of Finite Sampling. In the above experiments, our framework is trained using expectation values of measure- ment outcome computed via classical simulation, which corresponds to the ideal setting of infinite measurement samples. However, real-world experiments only allow sam- pling the states for finite times. To assess the robustness of our model under realistic conditions, we simulate finite- sample effects by introducing measurement inaccuracies at test time.
After learning the circuit representations for quantum states, it is natural to investigate the interpretability of these repre- sentations. To this end, we consider Hamiltonian learning as a downstream task to evaluate their effectiveness. Hamil- tonian learning is a task determine the parameters of an unknown Hamilonnian, which serves meaningful bench- mark for assessing how well the learned representations encodes the information of the underlying physical system.
In our setting, we use the circuit representations of the ground states to learn the corresponding Hamiltonians. The quantum systems we consider are the Ising model and the
The framework is first trained on simulation of infinite sam- pling data m = Tr(Mp) p) given measurement operator M and state p. At test time, we use finite measurement shots
8

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
k (128, 256, 512, 1024) obtain the measurement data (m)x the input Ising evolution, Ising ground states, Heisenberg ground states are 10, 50, 50, 10 respectively. The results are shown in Figure 7. Inaccurate measurement has nearly no effect on learning IQP circuits, where the action space contains no continuous parameters. For the other three families of states, using only 512 measurement shots is enough for high fidelity reconstruction, demonstrating the effectiveness of our framework in practical scenarios.
noise, strategies like error correction (Shor, 1995; Fowler et al., 2012) or error mitigation (Giurgica-Tiron et al., 2020; Liao et al., 2025) can be employed.
framework. System sizes for IQP,
(a)
(b)
0.9
0.8-
0.8
Global
0.6-
(a)
1.0
Fidelity
0.5
0.05
0.10
0.15
0.20
0.5
0.05
0.10
0.15
0
(c)
08
1.0
1.0
0.8-
0.8-
Global
Fidelity
0.2
Local
0.7
0.7
(c)
512
(d)
120 236
0.5
0:15
0.5
0:10
0.20
0.8
Noise strength
0.6
0.6
Figure 8. Impact of global depolarizing noise on the performance of our framework. (a) Learning states generated by IQP circuits. (b) Learning states evolved by Ising Hamiltonians. (c) Learning Ising ground states. (d) Learning Heisenberg ground states.
0.4
128 256
512
1024 128 256
512
1024
measurement shots
Figure Results under finite sampling conditions. (a) Learning states generated by IQP circuits. (b) Learning states evolved by Ising Hamiltonians. (c) Learning Ising ground states. (d) Learning Heisenberg ground states.
4. Conclusion and Outlook
We propose the explicit circuit representations, which fea- ture efficient property estimation and experimental recon- struction of quantum states. To learn this representation, we design reinforcement Icarning framework with Trans- former feature aggregation block and novel local fidelity reward function. The learning procedure relies exclusively on local measurement data, achieving high accuracy on pre- dicting properties. The learned representations can further be transferred to quantum systems of varying sizes and ap- plied Hamiltonian learning downstream task using linear model.
Impact of Circuit Noise. Real-world quantum circuits are affected by noise, which causes deviations between actual and ideal measurement outcomes. Unlike measurement inaccuracy, this gap cannot be mitigated via increasing the number of measurement shots. Therefore, it is important to investigate the impact of circuit noise to the construction procedure of circuit representations.
We evaluate the performance of our framework under the condition that the quantum circuit is affected by global depolarizing noise. The noisy output state is = (U))0 (i), where N represents the noise channel, U is the noise-free circuit. We set the noise parameter associated with the noise strength of to E (0.05, 0.1, 0.15, 0.2). System sizes for IQP, Ising evolution, Ising ground states, Heisenberg ground states are 10, 50, 50, 10 respectively. Figure illustrates the impact of varying noise strengths both global and local fidelity between the learned and target quantum states. Although fidelity degrades with increasing noise strength, our framework maintains consistently high performance up to a noise level of 0.2, demonstrating ro- bustness to moderate levels of noise. To deal with strong
Outlook. Our current experiments focus on learning quan- tum states prepared by shallow circuits and ground states of one-dimensional Hamiltonians. future work, we would extend our framework to more complex scenarios, such as deeper circuits, two-dimensional Hamiltonians, and critical behaviors of many-body Hamiltonians. Additionally, we would generalize approach to quantum process learn- ing, enabling the model to handle tasks involving varying input-output state pairs. Another promising direction is to integrate the physics knowledge from large language mod- els into the agent, which may further enhance the learning capability and efficiency of the framework.

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Acknowledgements
Caro, M. C., Huang, H.-Y., Ezzell, N., Gibbs, J., Sorn- borger, A. T., Cincio, L.,, Coles, P. J., and Holmes, Z. Out-of-distribution generalization for learning quantum dynamics. Nature Communications, 14(1). 2023.
We thank the anonymous reviewers for their helpful feed- back and suggestions to improve this work. This work supported by the Ministry of Science and Technology, China (MOST2030) with Grant No. 2023200300600, the National Natural Science Foundation of China (NseCy/Rescarch Grants Council (RGC) Joint Research Scheme via Project N.HKU7007/24, the Guangdong Provin- cial Quantum Science Strategic Initiative (GDZX2303007 and GDZXX200001), and the Hong Kong Research Grant Council (RGC) through the General Research Fund (GRF)
Cerezo, M., Sone, A., Volkoff, T., Cincio, L., and Coles, P. J. Cost function dependent barren plateaus in shallow parametrized quantum circuits. Nature Communications, 12(1), 2021.
Chen, A. and Heyl, M. Empowering deep neural quantum states through efficient optimization. Nature Physics, 20 1481,, 2024.
grant 17303923.
Impact Statement
Cotler, J. and Wilczek, F. Quantum overlapping tomography. Physical Review Letters, 124: 2020.
This paper presents work whose goal is to advance the field of Quantum Computation through incorporating advanced Machine Learning techniques characterize quantum states. There many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Cramer, M., Plenio, M. B., Flammia, S. T., Somma, R., Gross, D., Bartlett, S. D., Landon-Cardinal, O., Poulin, D., and Liu, Y.-K. Efficient quantum state tomography. Nature Communications, 1(149):177, 2010.
Du, Y., Huang, T., You, S., Hsich, M.-H., and Tao, D. Quan- tum circuit architecture search for variational quantum algorithms. npj Quantum Information, 8(1), 2022.
References
Aaronson, S. Shadow tomography of quantum states. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, pp. 325-33333333 -38, 2018.
Du, Y., Yang, Y., Liu, T., Lin, Z., Ghanem, B., and Tao, Shadownet for data-centric quantum system learning. arXiv preprint arXiv:2308. 1290, 2023.
Akhtar, A. A., Hu, H.-Y., and You, Y.-Z. Scalable and Flexi- ble Classical Shadow Tomography with Tensor Networks. Quantum, 7:1026, 2023.
Farhi, E., Goldstone, J., and Gutmann, S. quantum approximate optimization algorithm. arXiv preprint arXiv: 1et::4111.4022 2014.
Anschuetz, R. and Kiani, B. T. Quantum variational algo- rithms are swamped with traps. Nature Communications, 13(1), 2022.
Fetter, A. and Walecka, J. Quantum Theory of Many- pArticle Systems. Dover Books on Physics. Dover Publications, 2003. ISBN 9780486428277. URL https://books: wwwgleooooo...ooooooooooooo..... com wtt. hkmb.kkkkooo-dd Owekf1s83b0c.
Atkins, and de Paula, J. Atkins' Physical Chemistry. OUP Oxford, 2010. ISBN 9780199543373. URL https://books google. com. hk hk/bookskid- BV6CAOAAQBAJ.
Bengio, Y., Ducharme, R., Vincent, P., and Janvin, C. A neural probabilistic language model. Journal of Machine Learning Research, 3:1537-1155, 2003.
Fowler, A. G., Mariantoni, M., Martinis, J. M., and Cle- land, A. N. Surface codes: Towards practical large-scale quantum computation. Physical Review A, 86(3), 2012.
Bittel, L. and Kliesch, M. Training variational quantum algorithms is NP-hard. Physical Review Letters, 127: 120502, 2021.
Friis, N., Marty, O., Maier, C., Hempel, C., Holzäpfel, M., Jurcevic, P., Plenio, M. B., Huber, M., Roos, C., Blatt, R., et al. Observation of entangled states of a fully controlled 20-qubit system. Physical Review X, 8(2)022012, 2018.
Bremner, M. J., Jozsa, R., and Shepherd, D. J. Classical simulation of commuting quantum computations implies collapse of the polynomial hierarchy. Proceedings of the Royal Society A: Mathematical, Physical and Engineer- ing Sciences, 467((116):459 2010.
Georgescu, I. M., Ashhab, S., and Nori, F. Quantum simula- tion. Reviews of Modern Physics, 86:153-185 2014.
Giurgica- -iron, T., Hindy, Y., LaRose, R., Mari, A., and Zeng, W. Digital zero noise extrapolation for quantum error mitigation. In 2020 IEEE International Conference on Quantum Computing and Engineering (QCE), 2020.
Carleo, G. and Troyer, M. Solving the quantum many-body problem with artificial neural networks. Science, 355 (6325):602-606, 2017.
10

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Gross, D., Liu, Y.-K., Flammia, S. T., Becker, S., and Eisert, J. Quantum state tomography via compressed sensing. Physical Review Letters, 105:150401, 2010.
McClean, J. R., Boixo, S., Smelyanskiy, V. N., Babbush, R., and Neven, H. Barren plateaus in quantum neural network training landscapes. Nature Communications, 9 (I), 2018.
Guo, Y. and Yang, S. Quantum state tomography with locally purified density operators and local measurements. Communications Physics, 7(1), 2024.
McMillan, W. L. Ground state of liquid he" Phys. Rev. 138:A442-A451, 1965.
Huang, H.-Y., Kueng, R., and Preskill, J. Predicting many properties of quantum system from very few measure- ments. Nature Physics, 1050-10755 2020.
Mitarai, K., Negoro, M., Kitagawa, M., and Fujüi. K. Quan- tum circuit learning. Physical Review A, 98(3), 2018.
Motta, M., Sun, C.,, Tan, A. T., O'Rourke, M. J., Ye, E., Minnich, A. J., Brandao, F. G., and Chan, G. K.-L. De- termining eigenstates and thermal states on a quantum computer using quantum imaginary time evolution. Na- ture Physics, 16(2):205 210, 2020.
Huang, H.-Y., Chen, S., and Preskill, J. Learning to predict arbitrary quantum processes. PRX Quantum, 4:040337, 2023.
Huang, H.-Y., Liu, Y., Broughton, M., Kim, I., Anshu, A., Landau, Z., and McClean, J. R. Learning shallow quan- tum circuits. In Proceedings of the 56th Annual ACM Symposium on Theory of Computing, pp. 1343-1351, 2024.
Nielsen, M. and Chuang, I. Quantum Computation and Quantum Information: 10th Anniversary Edi- tion. Cambridge University Press, 2010. ISBN 9781 139495488. URI. https: //oops google. com. akk/booksiict s4DEy70 a0c.
Hubig, C., McCulloch, I. P., and Schollwöck, U. Generie construction of efficient matrix product operators. Physi- cul Review B, 95:035129, 2017.
Ostaszewski, M., Trenkwalder, I. M., Masarczyk, W., Scerri, E., and Dunjko, V. Reinforcement learning for optimization of variational quantum circuit architectures. In Proceedings of the 35th International Conference on Neural Information Processing Systems, volume 34, pp. 18182-18194, 2021.
Kitaev, A. Y. Quantum measurements and the abelian stabi- lizer problem. arXiv preprint "xXi::quant-ph/95/1 1026, 1026, 1995.
Kurmapu, M. K., Tiunova, V., Tiunov, E., Ringbauer, M., Maier, C., Blatt, R., Monz, T., Fedorov, A. K., and Lvovsky, Reconstructing complex states of a 20-qubit quantum simulator. PRX Quantum, 4(4):040345, 2023.
Perez-Garcia, D., Verstraete, F., Wolf, M. M., and Cirac, J. I. Matrix product state representations. arXiv preprint arXiv:quant-hh/6608/9777 2007.
Lanczos, C.
An iteration method for the solution of the eigenvalue problem of linear differential and integral op- erators. Journal of Research of the National Bureau of Standards, 45(4):255, 1950.
Peruzzo, A., McClean, J., Shadbolt, P., Yung, M.-H., Zhou, X.-Q., Love, J., Aspuru- Guzzik A., and O'Brien, J. A variational eigenvalue solver on a photonic quantum processor, Nature Communications, 5(1), 2014.
Lanyon, B. P., Maier, C., Holzäpfel, M., Baumgratz, T., Hempel, C., Jurcevic, P., Dhand, I., Buyskikh, A., Da- ley, A. J., Cramer, M., al. Efficient tomography of quantum many-body system. Nature Physics, 13(12): 1158-1162, 2017.
Qian, Y., Du, Y., He, Z.,, Hsieh, M.-H., and Tao, D. Mul- timodal deep representation learning for quantum cross- platform verification. Physical Review Letters, 133: 130601, 2024.
Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., and Dormann, N. baseliebsseiness:: Reliable rein- forcement learning implementations. Journal of Machine Learning Research. 22(268): 2021.
Larocca, M., Thanasilp, S., Wang, S., Sharma, K., Biamonte, J.. Coles, P. J., Cincio, L., McClean, J. R., Holmes, Z., and Cerezo, M. Barren plateaus in variational quantum computing. Nature Reviews Physics, 7(4): 2025.
Rényi, A. On measures of entropy and information. In
Lewis, L., Huang, H.-Y., Tran, V. T., Lehner, S., Kueng, R., and Preskill, J. Improved machine learning algorithm for predicting ground state properties. Nature Communica- tions, 15(1 (1)58995, 2024.
Proceedings of the fourth Berkeley symposium on mathe- matical statistics and probubility, volume contributions to the theory of statistics, volume 4, pp. 547-562 1961.
Sachdev, S. Quantum Phase Transitions. Cambridge University Press, 1999. ISBN 9780521582544, URL hetps://booke .ooisks.ooole...oooo com, hk/ booksookks? K_IhogAACAAJ.
Liao, M., Zhu, Y., Chiribella, G., and Yang, Y. Noise- agnostic quantum error mitigation with data augmented neural models. npj Quantum Information, 11(1):8, 2025.

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Sachdev, S. The quantum phases of matter. arXiv preprint 3XXiv:200344553 2012.
J. G., Laing, A., and Thompson, M. G. Experimental quantum hamiltonian learning. Nature Physics, 13(6): 551-555, 2017.
Scarpa, G., Molnár, A., Ge, Y., Garcfa-Ripoll, J.,, Schuch, N., Pérez-García, D., and Iblisdir, S. Projected entan- gled pair states: Fundamental analytical and numerical limitations. Physical Review Letters, 125:210504, 2020.
Wauters, M. M., Panizon, E., Mbeng, G. B., and Santoro, G. E. Reinforcement-leaminn aasinig---sssee tion. Physical Review Research, 2:033446, 2020.
assisted quantum optimiza-
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.066 2017.
White, S. R. and Feiguin, A. E. Real-time evolution us- ing the density matrix renormalization group. Physical Review Letters, 93:076401, 2004.
Wiebe, N., Granade, C., Ferrie, C., and Cory, D. Hamilto- nian learning and certification using quantum resources. Physical Review Letters, 112(19), 2014.
Sharir, O., Levine, Y., Wies, N., Carleo, G., and Shashua, A. Deep autoregressive models for the efficient variational simulation of many-body quantum systems. Physical Review Letters, 1 124(2), 2020.
Wu, W., Yan, G., Lu, X., Pan, K., and Yan, J. Quan- tumDARTS: differentiable quantum architecture search for variational quantum algorithms. In Proceedings of the 40th International Conference on Machine Leurning, 2023a.
Shor, P. W. Scheme for reducing decoherence in quantum computer memory. Physical Review A, 52:R2493 R2496, 1995.
Sorella, S. Green function monte carlo with stochastic reconfiguration. Physical Review Letters, 11:4558-45611 1998.
Wu, Y.-D., Zhu, Y., Bai, G., Wang, Y., and Chiribella, G. Quantum similarity testing with convolutional neural net- works. Physical Review Letters, 130:210601, 2023b.
Suzuki, M. Decomposition formulas of exponential op-
Wu, Y.-D., Zhu, Y., Wang, Y., and Chiribella, G. Learning quantum properties from short-range correlations using multi-task networks. Nature Communicationss 15(1): 8796, 2024.
erators and lie exponentials with some applications to quantum mechanics and statistical physics. Journal of Mathematical Physics, 26(4):601-61 1985.
Tang, Y., Xiong, H., Yang, N., Xiao, T., and Yan, J. To- wards LLM4QPE: Unsupervised pretraining of quantum property estimation and a benchmark. In The Twelfth International Conference on Learning Representations, 2024a.
Xiao, T., Fan, J., and Zeng, G. Parameter estimation in quantum sensing based on deep reinforcement learning. npj Quantum Information, 8(1):2, 2022.
Yao, J., Lin, L., and Bukov, M. Reinforcement learning for many-body ground-state preparation inspired by counter- diabatic driving. Physical Review X, 11:031070, 2021.
Tang, Y., Yang, N., Long, M., and Yan, J. SSL4Q: Semi- supervised learning of quantum data with application to quantum state classification. In Forty-first International Conference on Machine Learning, 2024b.
Zhang, S.-X., Hsich, C.-Y., Zhang, S., and Yao, H. Differ- entiable quantum architecture search. Quantum Science and Technology, 7(4):045033, 2022.
Torlai, G., Wood, C. J., Acharya, A., Carleo, G., Car- rasquilla, J., and Aolita, L. Quantum process tomography with unsupervised learning and tensor networks. Nature Communications, 14(1): (42588888, 2023.
Zhang, Y.-H. and Di Ventra, M. Transformer quantum state: A multipurpose model for quantum many-body problems. Physical Review B, 107(7), 2023.
Zhou, Z., Du, Y., Yin, X.FF., Zhao, S., Tian, X., and Tao, D. Optical quantum sensing for agnostic environments via deep learning. Physical Review Research, 6(4):043267, 2024.
Tóth, G., Wieczorek, W., Gross, D., Krischek, R., Schwem- mer, C., and Weinfurter, H. Permutationally invariant quantum tomography. Physical Review Letters, 105: 250403, 2010.
Zhu, Y. Wu, Y.-D., Bai, G., Wang, D.-S., Wang, Y., and Chiribella, G. Flexible learning of quantum states with generative query neural networks. Nature communica- tions, 13(1):6222, 2022.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u. and Polosukhin, Atten- tion is all you need. In Advances in Neural Information Processing Systems, volume 30, pp. 6000-6010, 2017.
Zhu, Y., Wu, Y.D.., Liu, Q., Wang, Y., and Chiribella, G. Predictive modelling of quantum process with neural net- works, arXiv preprint ar0iv:2308.0 X30:2200.088/5, 2023.
Wang, J., Paesani, S., Santagati, R., Knauer, S., Gentile, A. A., Wiebe, N., Petruzzella, M., O' O'Brien, L.., Rarity,
12

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
A. Related Work
Tomography-base quantum state characterization. Tomography-basee methods use direct measurement to characterize quantum states. To accurately characterize the full quantum state. Quantum State Tomography (Tóth et al., 2010; Gross et al., 2010; Cramer et al., 2010; Lanyon et al., 2017: Cotler & Wilczek, 2020) has been proposed, which measures the state in exponential number of basis obtain the state vector. Other methods focus constructing partial knowledge of the state. For instance, Shadow Tomography (Aaronson, 2018) targets at characterizing the measurement values of 2-outcome measurements using only few copics of the states. Classical shadow (Huang et al., 2020: Akhtar et al., 2023) utilizes randomized measurement to efficiently estimate local properties of the states. Noteworthy, there is a special family of work that uses Tensor network, e.g., Matrix Product State (MPS) GPerez-Garcia et al., 2007) and Projected Entangled Pair States (PEPS) (Scarpa et al., 2020), to approximate the state vector of quantum state. The original high dimensional state vector decomposed into multiple low-rank tensors with restricted bound dimension.
Bariational-basee quantum state characterization. Alternative to state tomography, variational quantum algorithms optimize the parameters of variational ansatz, i.e., parameterized quantum circuit, to approach the target state. Two representative methods are Variational Quantum Eigensolver (VQE) (Peruzzo et al., 2014) and Quantum Approximate Optimization Algorithm (QAOA) (Farhi et al., 2014). These methods update their output towards the target states, usually the ground states of a Hamiltonian, by measuring the energy and computing quantum gradient descend via, c.g., parameter shift rule (Mitarai et al., 2018). In addition to optimizing parameters, Quantum Architecture Search has been proposed to optimize the circuit ansatz. Du et al. (2022) traverse a candidate gate and select the gate configurations that achieve the highest scores the target objective. Wauters et al. (2020); Yao et al. (2021); Ostaszewski et al. (2021) utilize reinforcement learning to optimize the circuit while keeping quantum gradient descend to update parameters. Zhang et al. (2022); Wu et al. (2023a) propose differentiable strategy to simultaneously update the ansatz and parameters.
Machine learning- quantum state characterization. Machine learning can be used to learn the measurement values of states, and predict state properties. The machine learning state characterization methods can mainly be categorized into two classes - Neural Quantum State (Carleo & Troyer, 2017; Sharir et al., 2020; Zhang & Di Ventra, 2023; Chen Heyl, 2024) and Neural State Representation (Zhu al., 2022; Tang al., 2024a; Qian et al., 2024). The Neural Quantum State represents quantum state as neural network, where sampling the neural network corresponds to measuring the state. Parameters of the neural network can be updated via Variational Monte Carlo (McMillan, 1965) and Stochastic Reconfiguration (Sorella, 1998) methods. The Neural State Representation compresses the quantum state into a classical description, usually a low dimensional vector, via pretraining. Zhu et al. (2022) adopt a self-supervised manner to predict the measurement values of some measurement operators given other operators. Tang et al. (2024a) use language modeling (Bengio et al., 2003) as the pretraining strategy. Qian et al. (2024), the vector pretrains the representation by fitting the inner product to fidelity. Afterwards, the pretrained representation can be fine-tuned for downstream tasks, such as predicting the properties of quantum states.
Different from previous machine learning- dearning-based methods, we decode the state representation into a novel circuit representa- tion instead of low dimensional vector to support experimental reconstruction ability. Our representation is suitable for downstream applications like Hamiltonian learning. Unlike the reinforcement learning for quantum architecture search, our framework circumvents the need of caleulating gradients with respect to the circuit parameters, and possesses the ability to characterize family of states rather than one specific state.
B. Preliminaries
We review some of the key concepts in quantum computation. For a more comprehensive overview, please refer to Nielsen & Chuang (2010).
Quantum states are quantum counterparts of classical bits. They can be mathematically represented as vectors in Hilbert space, i.e., state vectors, denoted as (ı)) E Ca* satisfying l1 = 1, where N is the system size or the number of qubits. The notation| I-) is just used to emphasize that zp is a (column) vector. Its dual (row vector) is given by (. = |.>t, where "1" is the notation for conjugate transpose. The standard basis for quantum states is the computational basis {1 )22 where (i) is the vector whose i-th element is and others are O. For example, [0) = (1, O, o................. 0). Alternatively, we can use the mixed state to describe a probability ensemble of quantum states {p,, (().) P; is the probability of the quantum system being in the state loh)). This can be represented as density matrix p C2 x2" where م E, Pa (bs) (vhel. Clearly, for pure state [17), the corresponding density matrix is (1b). Multiple quantum states can be combined to form compositional system, which is
13

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
represented by the tensor product (Kronecker product) denoted as "@". For two states (1),, (o) E C²* their composition
given by (8) & (o) E c We use the notation 1-y6N to denote an N-qubit product state, e.g., 10),0N = (0) @ @ 10).
The similarity between two quantum states can be quantified by (global) fidelity and trace distance. In this paper, we focus exclusively on the global fidelity. Given two density matrices p and a. the global fidelity is defined
F(p, .) (Tr (Vot)/opp/12)
If the two states are pure states (vb) and (o), the fidelity simplifies to (+)lo) which is closely related to the cosine similarity between two vectors.
Quantum states can be measured, causing them to collapse into classical bits. Measurement is described by a set of measurement operators [Mj}. where each Mj is a Hermitian matrix, i.e., M, My. In the case of projective measurements, the operators are projectors that satisfy Σ; M, I and M, = 6j. The measurement outcomes, which correspond to classical bits, are associated with the index j. When measuring a state p. the probability of obtaining outcome, j is given by p(j) = Tp) The observable M = ΣS JM describes the overall measurement results, and the expectation value of the measurement on the state is m = E; jP(j) = pr. Additionally, measurement operators can be composed using tensor products to form new measurements for larger quantum systems.
Quantum states can be evolved by quantum gates, analogous to classical logical gates, which are represented by unitary matrices U that satisfy UTU UU T. A unitary matrix can be generated from a Hamiltonian H a Hermitian matrix using a parameter é, and is expressed as (()) = Y, and Z, where
A special group of unitary matrices are Pauli matrices X,
0 1 1 0 Y
0
0
x
Z
(10)
-1
The Pauli matrices form the single-qubit Pauli gates. Besides these gates, other typical quantum gates are single-qubit
rotation gates (0) exp(-ix0/2), (0) exp(-iX00/2 (0) = cxp(-iZ0/2) and two-qubit gates CX-
[0)(0] & I + (1) X, CZ = [0)(0] @ I [1] @ Z. More general quantum gates can be decomposed into these
single-qubit and two-qubit gates.
C. Proof of Proposition 2.1
Proposition 2.1 states that the agent learns a policy that constructs an N-qubit quantum state with average n-local
(1 S n S N) fidelity (00) (p5t), ,10) (0 (0 2 1 e, then the global fidelity satisfies F(p!") (0)os 2 1- -N Before proving this proposition, we present the following lemma.
Lemma C.1. The observable ((r) associated with the n-local fidelity has the largest eigenvalue Ao = 1 and the second
largest eigenvalue \ = 1 - 1/(N - n + 1).
Proof. The local operator o(n) acting on qubits f
+ i
1} can be expressed
o 1
(0) ii @ 1 18.../ diag (1, 1, .., 1.0,0.
(11)
N N-i-R
diag
(12)
= -diag 12,, Ogten 25 1 Ogttn
12: Ogctn
(13)
Now that 0²²)) is diagonal matrix, the elements 1s and Os are the eigenvalues. Next, we are interested in the eigenvalues of O(a), which is defined
N-n olm)
1
mn)
(14)
N
1=0
((n) is also a diagonal matrix that has eigenvalues 0 < A S 1, with corresponding eigenvectors (j). Each of Ay is a
sum of N- n + items of the corresponding j-th entries on the diagonal of local operators O."), denoted by 0.") (.")
14

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
( j S 2 - 1). Since the entries [)] are either 1 or O, the value of > depends on the number of in O [0]] Set
[j]
j = 0, we obtain the largest eigenvalue as all entries 0 [0] are I, thus Åo = 1. Next, let 1, the entry =²²) [1] is 0 while others are 1, so the second largest cigenvalue \ (N - -n)/(N - + 1) = 1 - (/ n 1).
Corollary C.2. Xu is a unique eigenvalue of O(m).
Proof. Consider 2 -1 1 for any 0 < < N - n + 1, we notice that ((()(]] = and 0 = 0. Thus there cannot
exist j > O in which the entries O [1] [i] are for all i. This means that j = 0 generates the unique largest eigenvalue
Au = 1 with eigenvector (Ao) = [0) e
Now we prove Proposition 2.1. We use spectral decomposition on the local fidelity observable O to construct the relation between average local fidelity Tr(O(m) p) and fidelity as follows
Tr(O Tr Σ
AelAa)
(15)
R20
(0 @N Tr
Σ Sell))
(16)
F + du (Aullleee)
(17)
F + (NlPPIA))
(18)
A1 - (HoloAAo))
(19)
= -F+A(11-F)
(20)
Lemma C.I tells us that A = N N-n-T Suppose (In) (0)(N) = Tr(O(")p) p) 2 1 - e, then F 2221 1 T-
1 - (N - n + 1)e.
D. Quantum Properties of Interest in the Experiments
In the experiment section, we consider 3 different properties along with global fidelity for performance evaluation, namely the second-order Rényi entropy (Rényi, 1961), two-point correlations (Fetter & Walecka, 2003) and spin-Z values (Atkins & de Paula, 2010). These are important quantities that characterize quantum states from different perspectives. Rényi entropy a non-linear property, while the two-point correlation and the spin-Z, are linear properties.
Second-order Rényi entropy. This quantity is used to characterize the subsystem (some of the qubits) entanglement of quantum state. Denote PA as the reduced density matrix of quantum state p on its subsystem A, i.e. PA = Tp) The Rényi entropy quantifies the entanglement strength of A, which is computed by
Sa(PA) = 1
log Tr(pA),
(21)
where a is the order, which set to 2 in our experiments. We consider the average value of N qubit subsystems.
Two-point correlation. The correlation function describes the relationships between different parts of the quantum system. This is useful for characterizing quantum phases of matter (Sachdev, 2012) and studying critical behavior (Sachdev, 1999). We consider the two-point correlation defined as follows
Co. = Z,pp.
(22)
We take the average of all correlation values for و < N.
15

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Spin-Z value. This quantity describes the angular momentum of many-body quantum state. In our experiments, we consider the spin-Z. value, namely the angular momentum in the Z direction, which is defined as
Tr Σ Zap
(23)
To evaluate the performance of different methods in predicting the aforementioned properties, we first compute the true properties of the target states. Next, we apply the benchmarked methods to predict these properties. Finally, we calculate the root mean squared error (RMSE) between the actual and predicted properties as the evaluation metric.
E. Simulation of Quantum Systems
To simulate large-scale quantum systems, we use the Matrix Product State (MPS) (Perez-Garcia et al., 2007) represent quantum states, rather than directly using the full state vector. MPS decomposes the state vector into a chain of low-rank tensors through methods such singular value decomposition, truncating the singular values to compress the state from ((2%) to (N)22) scale, where d is the physical dimension (typically d = 2 for qubit systems), and is the bond dimension, which represents the number of singular values retained and related to the degree of entanglement. For product states, x 1, while for maximally entangled state, X scales exponentially with the system size. Since the quantum states we consider exhibit low degree of entanglement, e.g., the Ising ground states, the Heisenberg ground states, and states prepared by shallow circuits, we restrict X S 16 throughout our experiments.
Afterwards, to simulate the evolution of states, we apply Matrix Product Operators (MPO) (Hubig et al., 2017) MPS. The evolution of quantum states can be viewed applying unitaries to the states, which is equivalent to applying MPO to MPS. For single-qubit gates, the MPO is simply the gate itself. For multi-qubit gates, the corresponding MPO can be derived through tensor decomposition similar to MPS. To simulate the time evolution of a state (ı)) governed by a Hamiltonian = >, Hi, where Hj are local Pauli terms, we first apply the first-order Trotter decomposition (Suzuki, 1985)
to approximate r This yields
N
iHo
(24)
where 5, is the time step and N = T/6,. In the Ising evolution experiment, we set 8, = 0.1. Following this, we use the Time-Evolving Block Decimation (TEBD) algorithm (White Feiguin, 2004) to simulate the evolution. The Hamiltonian terms are divided into even and odd components, and series of brickwork MPOs are applied the MPS to perform the time evolution.
For simulating the ground states, we use the DMRG algorithm. First, the Hamiltonian is decomposed into MPO. Then each tensor of MPS is iteratively updated, sweeping from left to right and from right to left. For each tensor, Lanczos method (Lanczos, 1950) is applied to find the eigenvalues and eigenvectors, and the tensor is optimized to the eigenvector with the minimum eigenvalue. This procedure is repeated until the energy converges. In our implementation, the MPS randomly initialized. We set the maximum dimension of Krylov space 10, and the maximum sweep steps 200. The iteration stops if the energy difference between to updates is smaller than 10-4, Note that for Hamiltonians with degenerate eigenspace, the ground states found by DMRG can be different for different initialization of MPS and different parameter specification. Therefore, we turn to imaginary time evolution (Motta et al., 2020) to simulate the Heisenberg ground states, which is steered by TEBD algorithm with the time being an imaginary number. This guarantees deterministic ground states the initial MPS, the time step 6, and total steps N are fixed. We set the initial MPS to (0), 8, = 0.01 and = 10.
F. Additional Implementation Details
The policy network of our agent consists of a Transformer measurement feature aggregation block as encoder followed by an MLP for decision making. The encoder comprises two Transformer encoder layer. The positional encoding follows the standard procedure in (Vaswani et al., 2017). The embedding dimension is set to 128. The number of heads for the MHA is 4. For the decision making MLP, we use 3 linear layers with ReLU activation, and the feature dimension is 512. We discard any dropout layer.
To train the policy network, we use Adam optimizer with learning rate 0.001. For the implementation of PPO, we use Stable
16

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
Baselines3 package (Raffin et al., 2021). The batch size is set to 1000. We cutoff KL divergence 0.05 between two updates of the policy enhance training stability.
G. Resource Requirement for Training and Inference of QCrep
Table 6 and Table 7 detail the resources required in training and inference for each experiment we conducted respectively. "System Size" denotes the number of qubits of the target state family. "#Iterations" denotes the total number of iterations required for the RL agent to learn the family states from beginning until convergence, where each iteration is an episode of maximum length T 200 for Ising ground states and 100 for others. "#Observables" is the number observables required for measurements in each iteration.
Table 6. Resource requirement for training.
Experiment
System Size
#lterations
#Observables
IQP
50
610
441
Evolve Ising
50
1240
441
Ground Ising
50
1880
441
Ground Heisenberg
10
2040
81
Table 7. Resource requirement for inference.
Experiment
System Size
Circuit Depth
#Observables
IQP
50
2
441
Evolve Ising
50
10
441
Ground Ising
50
22
441
Ground Heisenberg
10
28
81
H. Additional Experiment Results
H.1. Learning States Using n-local Fidelity with Larger n
With the increment of entanglement, the local fidelity and global fidelity diverges but still be lower-bounded by Proposition 2.1. In practice, one can increase n obtain higher global fidelity. We consider 50-qubit quantum states prepared by 10 layers of randomly selected single-qubit and two-qubit gates. The gates satisfy V = =exp(=xxdd exp(-iéG)) where ó E [- 2///2..z// T/2] and G are sampled from universal single- and two-qubit Pauli operators. The model is trained using n-local fidelity. Figure 9 shows the scaling of global and local fidelity with respect to n. The local fidelity remains high, indicating that the circuit representation can still be faithfully utilized to predict local properties of interest. Whereas training on 1-local fidelity, the global fidelity drops to nearly However, this can be improved if increase n, which attains relatively high value n 4. The result further demonstrates the versatility of our model on learning various types of quantum states.
H.2. Learning Rotated GHZ States
To achieve faithful global fidelity during circuit construction for states with higher entanglement, is typically necessary to use n-local fidelity with larger However, we demonstrate that in certain cases, high global fidelity be attained using only I-local fidelity, even for highly entangled states, by carefully designing the action space. Specifically, we consider family of Z-rotated GHZ states as the target states, which are known to be maximally entangled. We set the number of qubits N = 50. The action space is chosen as { H, CNOT, Rg (0)), with (-T/4, /4, π /4). As shown in Table 8, both the local and global fidelity between the reconstructed and target states indicate that the agent can still achieve strong performance, despite the high entanglement.
17

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
1.0
0.8
Fidelity
0.6
0.4
0.2
Global
Local
0.0
2
3
4
5
n
Figure 9. Learning 50-qubit states prepared by 10-layer universal circuits using n-local fidelity.
Table 8. Learning 50-qubit Z-rotated GHZ states.
Local Fidelity
Global Fidelity
0.9927 + 0.0151 0.9275 + 0.1511
H.3. Universal Gate Set as Action Space and Mixture State Family
In our experiments, we focus on restricted action spaces. They constructed by utilizing prior knowledge of the underlying physical system of the target state family. It is an interesting question to explore how the agent performs when a universal gate set is considered, and when the state family is not restricted to one particular physical system.
Here we consider a mixture state family - the ground states of Ising model together with the ground states of Heisenberg model. The coefficients of the Hamiltonian are chosen the same as Experiment Section 3.3. We the number of qubits 4. The gate set is chosen as g exp(i0G), where G {X, Y, 2) U {x, Y, Z] 02 takes all possible combinations of single- and two-qubit Pauli operators, which form universal 2-local gates. The parameters & [-m/2, x /2]. Table 9 shows that our model can also perform well using universal gate set. We highlight that in many practical scenarios, some prior information is available to inform the choice of action space. For example, it is often possible to learn the ground states of berisenberg-interaciie many-body system without knowing the interaction coefficients but knowing the skeleton of the Hamiltonian.
Table Learning mixture state family using universal 2-local gates.
Experiment
System Size
Fidelity
Rényi Entropy
Two-point Correlations
Spin-Z
Mixture family
4
0.9587+0.0130
0.0745
0.0128
0.0434
Ablation Study
In our framework, the agent employs Transformer network as feature extractor for the measurement data. A simpler alternative is to use an MLP. In this section, we investigate the impact of architectural choices on entanglement capture and zero-shot transferability by considering the task of learning 50-qubit Ising ground states. First, we replace the Transformer block with an MLP. The input measurement data is flattened and passed through the MLP with 49 x 9 input neurons. Unlike the Transformer, this architecture is not size-agnostic, and the number of input neurons increases with system size. As
18

---

Reinforeed Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements
result, the MLP architecture does naturally support transfer systems of different sizes. To enable zero-shot transfer, pad the measurement data with zeros when applying the ML.P-triined model 10-qubit system.
Results in Table 10 shows that, while the local fidelity achieved by the Transformer and MLP are comparable, the Transformer outperforms the MLP in terms of global fidelity. This suggests two key advantages of the Transformer architecture in our framework: (1) it effectively captures entanglement as long-range dependencies in the measurement data, without increasing the number of network parameters with system size, and (2) enables zero-shot transfer learning across quantum systems of different sizes.
Table 10. Architectural comparison of feature extraction layer.
Model
System Size
Local Fidelity
Global Fidelity
Transformer
50
0.9986 + 0.0003
0.9673 + 0.0083
Transformer (transfer)
10
0.9876 = 0.0032
0.9391 = 0.0160
MLP
50
0.9985 + 0.0002
0.9560 + 0.0041
MLP (transfer)
10
0.7311 = 0.0271
0.1617 + 0.0456
19


Reviews:
[{"review_id": "kqXpV3l9oG", "reviewer": "Reviewer_5m67", "summary": "This paper introduces QCrep, a novel reinforcement learning framework for quantum state characterization that generates explicit circuit representations rather than implicit neural encodings. The innovation lies in using local measurements from neighboring qubits to learn circuit descriptions that can faithfully reconstruct quantum states of interest. This represents a significant departure from existing approaches that either require exponentially many measurements or produce black-box neural representations that lack physical interpretability.\n\nThe authors developed a transformer-based measurement feature aggregation architecture to extract global quantum features from local measurement data, coupled with a local fidelity reward function that mitigates the notorious barren plateaus problem. Their theoretical analysis establishes a formal relationship between local and global fidelity, providing mathematical justification for their approach. Empirically, they demonstrate QCrep's effectiveness on diverse quantum states up to 100 qubits, including IQP circuit states, time-evolved Ising model states, and many-body ground states, while also showing its utility for downstream tasks like Hamiltonian learning.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. W1. Exploration in highly entangled states.**\n\nAlthough fully reconstructing the target states for highly entangled states using local fidelity is generally difficult, our framework would work if the state characterization task is to estimate some properties of interests like correlations, which is the primary focus of our proposal. High local fidelity can be obtained even if the target states have higher entanglement like the states generated from random brickwork-like circuits in section 4. Besides, full reconstruction of highly entangled states is not entirely out of reach. For example, prior knowledge on how target states are formed can be leveraged to guide the agent’s learning process. Please refer to reply 4 to Reviewer 9UgW where we learn random rotated GHZ states.\n\n**2. W2. Computational complexity analysis on training cost.**\n\nThe training cost depends on the number of observables, the maximum episode length $T$ and the iterations required for convergence. The first two components are analyzed in the main text, where the number of observables scales linearly with the system size regardless of entanglement; 1024 total measurement shots suffice to estimate the expectation values of local observables, and $T$ is fixed for each family of states regardless of system size. The convergence rate, however, is more difficult to analyze theoretically. We provide empirical evidence using the task of learning ground states of the Ising model. We train the agent to learn on 10- and 50-qubit states. The total number of iterations are set to 1740 and 1880. The agent achieves average global fidelity 0.9691 and 0.9673 respectively. This result aligns with the zero-shot transferability of the agent across different system sizes.\n\n**3. W3. Additional ablation studies on the model architecture.**\n\nThe policy network is composed of one transformer encoder and one MLP for decision making. For the impact of the transformer encoder, please refer to the reply 2 to Reviewer 9UgW.\n\n**4. Q1. Performance degrade with increasing entanglement entropy.**\n\nFor estimating local properties, e.g., Renyi Entropy and two-point correlations, entanglement does not matter much, since the agent can approach high local fidelity for higher entangled states like Figure 7. The degradation of global fidelity is related to the entanglement entropy, if one chooses 1-local fidelity as reward function and chooses a relatively general action space. Consider the Ising evolution as an example, where the circuit depth grows with the evolution time thus entanglement increases. The following table records the scaling of global fidelity with respect to evolution time $t$. The system size is 50 qubits.\n\n| t    | Fidelity |\n| ---- | -------- |\n| 1.2  | 0.9933   |\n| 3.4  | 0.9934   |\n| 5.6  | 0.9816   |\n| 7.8  | 0.6284   |\n\nThe performance stays **stable for a relatively long time** before degradation. This is primarily because the bond dimension of the Tensor Network is not enough to accurately represent the state, which is a **limitation of the simulation** rather than our agent. In practice, if we can sample from a large scale quantum computer, we would expect better results.\n\n**5. Q2. Exploring other reinforcement learning algorithms.**\n\nPPO is a **robust and scalable** method that has successfully guided LLM across corpora comprising billions of texts [1], making it a good choice for learning quantum states. While off-policy methods could be explored, the potential inefficiency caused by inferior history actions remains a concern. As for model-based reinforcement learning, our framework already aligns with this paradigm. The simulator we use to evolve the quantum states could be regarded as a surrogate to the real quantum device.\n\n[1] Ouyang, Long, et al. \"Training language models to follow instructions with human feedback.\" Advances in neural information processing systems 35 (2022): 27730-27744.\n\n**6. Q3. Influence of higher n-local fidelity rewards on results and computational costs.**\n\nHigher n-local fidelity will result in barren plateaus problem thus the agent becomes harder or even unable to train if $n$ is too large. The number of observables required scales exponentially with $n$. However, for smaller $n$, e.g., $n\\leq 5$, increasing $n$ would help obtain a higher global fidelity in practice, as shown in Figure 7.\n\n**7. Q4. Extendability to directly optimizing specific quantum properties instead of state fidelity.**\n\nYes. Many properties can be computed by measuring the quantum state using some specific observables like Pauli X, Y and Z. Since the neural network agent can be optimized using local fidelity, obtained by measuring subsystems of the state with Pauli Z, we anticipate that the agent can also be optimized if changing Z to other observables that corresponds to some specific properties of interest.\n\n**8. Q5. Perform on mixed states.**\n\nPlease refer to the reply 4 to Reviewer cAXj.", "claims_and_evidence": "The paper's primary claims are generally supported by their theoretical analysis and experimental results:\n\nThe claim that QCrep can learn explicit circuit representations for quantum states using only local measurements is convincingly demonstrated across multiple experiments. The authors show they can achieve high fidelity with only $O(N)$ observables instead of the exponential number typically required.\n\nThe authors claim their reinforcement learning approach with local fidelity rewards avoids barren plateaus. This is supported indirectly by their ability to scale to systems much larger (50-100 qubits) than what's typically achievable with gradient-based methods, though direct landscape analysis would strengthen this claim.\n\nTheir claim regarding zero-shot transfer to different system sizes is compellingly validated by their experiments showing a model trained on 50-qubit systems can generalize to systems ranging from 10 to 100 qubits.\n\nThe usefulness of the circuit representations for downstream tasks is demonstrated through Hamiltonian learning experiments, where the learned representations enable parameter prediction with high accuracy.\n\nThe paper's evidence for robustness to finite sampling and noise (in the appendix) supports practical applicability, showing the method works with as few as 512 measurement shots and moderate levels of depolarizing noise.\n\nOne claim that could use more thorough validation concerns the method's performance on highly entangled states, as the current experiments focus on states with moderate entanglement.", "methods_and_evaluation_criteria": "The paper's methodological approach represents a creative combination of techniques from quantum computing and machine learning. The key insight is inverting the typical state preparation problem: rather than learning circuits that prepare target states from $|0⟩^{\\otimes N}$, they learn circuits that evolve target states toward $|0⟩^{\\otimes N}$. This enables handling multiple states from a family.\n\nThe evaluation criteria are comprehensive and appropriate:\n\n1. Global and local fidelity measure how well the reconstructed states match targets\n2. Renyi entropy evaluates the reconstruction of entanglement properties\n3. Two-point correlations assess how well quantum correlations are captured\n4. Spin-Z values verify the reproduction of local observables\n\nTheir comparative analysis against TQS, VQE, QAOA, and QAS consistently demonstrates QCrep's superior performance across all metrics, especially for larger systems.\n\nThe ablation studies examining finite sampling effects and circuit noise are crucial for assessing practical viability, though these results would benefit from inclusion in the main paper rather than the appendix.", "theoretical_claims": "The paper's theoretical foundation is sound, particularly Proposition 2.1. This provides a mathematical justification for using local fidelity as a reward function.\n\nThe proof using spectral decomposition of the local fidelity observable is correct and insightful, showing why local optimization can yield good global properties - a question of fundamental importance in quantum many-body physics.\n\nThe theoretical analysis connects to broader questions about the information content of local reduced density matrices and the conditions under which they uniquely determine global states.\n\nThe paper doesn't explicitly analyze the representational power of bounded-depth quantum circuits, which would strengthen the theoretical foundation given the empirical success of relatively shallow circuits for complex states.", "experimental_designs_or_analyses": "The experimental design is comprehensive, covering diverse quantum state families:\n\nFor each experiment, the authors clearly describe the system configurations, parameters, and evaluation metrics. The comparisons with baseline methods are fair and thorough.\n\nThe zero-shot transfer experiments demonstrating generalization across system sizes are particularly valuable, as is the out-of-distribution generalization test for Heisenberg ground states.\n\nThe Hamiltonian learning experiments effectively showcase the downstream utility of the circuit representations, showing they encode physically meaningful information that can be extracted with simple linear models.\n\nThe appendix experiments on universal gate sets and mixed-state families demonstrate the method's flexibility beyond the specific configurations in the main paper.\n\nAdditional analysis of how performance scales with entanglement complexity would strengthen the experimental component.", "relation_to_broader_scientific_literature": "The paper distinguishes itself from neural quantum states (like GQNQ and NQS) by producing explicit rather than implicit representations, addressing a key limitation of previous ML approaches.\n\nThe connection to quantum process tomography could be elaborated further, as circuit representation learning has parallels to process reconstruction.", "essential_references_not_discussed": "The paper's literature review is thorough, but more discussion on relation to NQS could be beneficial", "other_strengths_and_weaknesses": "Weaknesses:\n\nThe paper focuses primarily on states with moderate entanglement. The limits of the approach for highly entangled states remain unclear.\n\nThe computational complexity analysis could be more thorough, particularly regarding how training costs scale with system size and entanglement.\n\nThe paper doesn't fully explore which architectural components contribute most to performance. Additional ablation studies on the model architecture would provide deeper insights.", "other_comments_or_suggestions": "No other comments.", "questions_for_authors": "1. How does performance degrade with increasing entanglement entropy of target states? Is there an entanglement threshold beyond which the method becomes ineffective?\n2. is it worth exploring more sophisticated reinforcement learning algorithms beyond PPO? Off-policy or model-based RL might further improve sample efficiency.\n3. How would using higher n-local fidelity rewards affect results and computational costs? Your theory suggests this could tighten global fidelity bounds.\n4. Could the framework be extended to directly optimize for specific quantum properties rather than state fidelity?\n5. How does the method perform on mixed states rather than pure states? Many experimental quantum systems produce mixed states due to decoherence.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xxhJG7qDFi", "reviewer": "Reviewer_cAXj", "summary": "This work develops explicit quantum state representations by generating surrogate preparation circuits through reinforcement learning. The approach uses a local fidelity reward function and a quantum measurement feature aggregation block that extracts global features from local measurement data. The paper attempts to establish theoretical analysis of the relationship between local approximation and global fidelity. Experiments demonstrate successful learning of special classes of quantum states up to 100 qubits.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. W1 & Q1. The accuracy-sample cost tradeoff, estimation of density matrix fidelity.**\n\nThe impact of finite sampling (inaccurate expectation value estimation) to the accuracy has been discussed in Appendix G1.\n\nWe would like to clarify that while estimating density matrix fidelity is difficult in general, our framework ensures accurate and efficient estimations due to following reasons:\n\n(1) In our framework, instead of evolving the product state 0 towards the target state, we construct circuits to evolve the target state to 0. The two approaches are equivalent because quantum gates are invertible, but our approach only needs to estimate the **local** fidelity between an arbitrary state (density matrix) and 0. This can be done efficiently because it only requires measuring Pauli Z observables.\n\n(2) We only need to compute the local fidelity rather than global fidelity as the reward function for training, which can be estimated by applying local Pauli Z operators to the output state, demonstrating the practical implementation of our framework.\n\n**2. W2 & Q2 & Q5. More implementation details.**\n\n(1) For the **algorithm implementation**, the policy network of the agent contains a 2-layer 4-head Transformer encoder with hidden dimension 128. The positional encoding follows the standard procedure in [1]. The final MLP has 3 linear layers with ReLU activation, and the feature dimension is 512. \n\n(2) The **circuit ansatz design** has been detailed in section 2.3 and section 3, the circuit layout is brickwork shown in Figure 1.\n\n(3) The **input assumptions** to the network has been discussed in section 2.2, where we exclusively measure expectation values of two-local Pauli observables, i.e., XX, XY, YX, YY, YZ, ZX, ZY, ZZ. The impact of inaccurate measurement is discussed in Appendix G1, showing that 1024 measurement shots is enough to obtain a relevantly accurate estimation of expectation values and attain no degrade of performance.\n\n(4) To **optimize** the policy network, we use Adam optimizer with learning rate 0.001. We use stable baselines3 [2] for the implementation of PPO. The batch size is set to 1000. We set a cutoff KL divergence 0.05 between two updates of the network to enhance training stability.\n\n(5) The **numerical simulation details**, including the Tensor Network simulation of quantum systems, and the resource requirement for training and inference are presented in Appendix E and F.\n\n[1] Vaswani, Ashish, et al. \"Attention is all you need.\" Advances in neural information processing systems 30 (2017).\n\n[2] Raffin, Antonin, et al. \"Stable-baselines3: Reliable reinforcement learning implementations.\" Journal of machine learning research 22.268 (2021): 1-8.\n\n**3. W3 & Q3. Proofs for the large-scale quantum state learning.**\n\nPlease refer to the reply 3 to Reviewer 9UgW, where we briefly analyze that **even for learning product states, the global fidelity naturally degrades exponentially with the system size, but the performance of our framework remains stable when transferring to larger systems**.\n\n**4. W5 & Q4. Evidence of practicability and efficient implementability on near-term or future quantum hardware.**\n\nFirst, the measurement settings for our framework is efficient and practical that only require nearest-neighbor measurements, including the input data acquisition and the reward computation. These resources can be easily obtained in real experiments. Second, the action set only contains single-qubit or two-qubit local gates. This means that our framework is implementable even on a quantum hardware without long-term qubit connections. Moreover, our framework follows naturally the spirit of Sim-to-Real [1] learning. The agent could first be trained in the simulation environment, where the expectation values can be accurately and efficiently computed. Then the agent is transferred to a real quantum hardware to conduct state reconstruction.\n\nFurthermore, we have demonstrated the impact of circuit noise in Appendix G2, where the agent is first trained in noiseless environment and transferred to a quantum circuit suffered from depolarizing noise. The output states affected by the\nnoise become mixed states. The results shows that the agent can tolerate noise strength below 0.2 and maintain good performance. We anticipate this to be a natural advantage owing to the reinforcement learning pipeline. Because during training, rather than greedily selecting the action that maximizes the reward, the agent makes explorations in different gate combinations and gate parameters, thus becomes aware of how to adjust the action when the observation (the input measurement data) deviates from the ideal case.\n\n[1] Zhao, Wenshuai, Jorge Peña Queralta, and Tomi Westerlund. \"Sim-to-real transfer in deep reinforcement learning for robotics: a survey.\" 2020 IEEE symposium series on computational intelligence (SSCI). IEEE, 2020.", "claims_and_evidence": "The claims and evidence are basically fine. One concern is about the implementation of the algorithm, as the loss function involves calculation of the fidelity function of density matrices, which is difficult to estimate. The impact of samples and their inaccurate estimation on the overall method is unclear.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are less convincing due to insufficient description of the algorithm details, circuit ansatz, sample cost, and input assumptions.", "theoretical_claims": "There are no major proofs.", "experimental_designs_or_analyses": "The experimental design does not provide sufficient evidence to show whether the method is practical and efficiently implementable for near-term quantum computers.", "relation_to_broader_scientific_literature": "The relation to broader scientific literature is basically reasonable.", "essential_references_not_discussed": "The reference is reasonable.", "other_strengths_and_weaknesses": "The work shows an interesting attempt for learning quantum states. But I have some concerns:\n1. The algorithm relies on fidelity calculations of density matrices, which are difficult to estimate accurately. As this is a key subroutine, the accuracy vs. sample cost of fidelity needs more analysis. A more careful analysis concerning the whole implementation is needed.\n2. When reading the paper, I feel that I need clearer details on the algorithm specifics, circuit ansatz design, optimization, sample complexity, and input requirements or assumptions.\n3. For the large-scale solution, the work lacks rigorous proofs to support its claims and theoretical analysis.\n4. Experimental results don't sufficiently demonstrate the method's practicality or efficient implementation on near-term quantum hardware.\n5. It would be better to provide the detailed methodology of numerical simulations for the experimental part.", "other_comments_or_suggestions": "The comments are shown above.", "questions_for_authors": "1. How does your algorithm handle the accuracy-sample cost tradeoff when estimating density matrix fidelity, and what analysis have you done to validate its practical implementation?\n2. Could you provide more specific details about your algorithm implementation, circuit ansatz design, optimization procedure, sample complexity requirements, and the input assumptions your method makes?\n3. What rigorous proofs support your theoretical claims, particularly for the large-scale quantum state learning solution you propose?\n4. What evidence demonstrates that your method is practical and efficiently implementable on near-term or future quantum hardware?\n5. Can you elaborate on the detailed methodology used for your numerical simulations (up to which level) in the experimental section?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q58f6SafUJ", "reviewer": "Reviewer_9UgW", "summary": "The paper uses a deep reinforcement learning algorithm to construct quantum circuits in a manner that avoids barren plateaus by not requiring gradients with respect to the circuit parameters. The approach appears to outperform alternative approaches, including VQE and QAQA.\n\nA nice addition to the work is the use of a transformer architecture that the authors explicitly tie to the modelling of entanglement between qubits in the circuit. Further, as the circuit is built from local rewards, it appears that the algorithm is scalable.\n\n## Update after rebuttal\n\nI was happy to see the rebuttal from the authors and was happy to raise my score. I am confident that the authors will update the relevant parts of the manuscript and it will make a great contribution to the conference.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Suggestion 1. Refinement of figures.**\n\nWe will adjust the legend position and font size. Current figures are in PDF format and can be enlarged without loss of resolution.\n\n**2. Q1. Benefits of transformer architecture. Capture of entanglement.**\n\nCompared to simpler architecture like MLP, the transformer in our framework has two main advantages: 1. It allows for capturing entanglement (long-term relation in the measurement data) **without increasing the number of parameters** of the neural network. 2. It enables transfer learning ability to different system sizes. We did further ablation study on learning 50-qubit Ising model to validate our claim. First, we use MLP to replace the transformer block. The input data is flattened and fed into MLP with $49\\times 9$ input neurons, which increases with the system size unlike the transformer that is size-agnostic. Note that this formulation doesn't directly allow transfer to a different system size since the architecture is fixed. Second, we perform zero-shot transfer learning of this MLP-based to 10-qubit system, by concatenating 0 to the measurement data. The table below records the performance.\n\n| Model                  | Local               | Global              |\n| ---------------------- | ------------------- | ------------------- |\n| Transformer            | 0.9986 $\\pm$ 0.0003 | 0.9673 $\\pm$ 0.0083 |\n| Transformer (transfer) | 0.9876 $\\pm$ 0.0032 | 0.9391 $\\pm$ 0.0160 |\n| MLP-fix                | 0.9985 $\\pm$ 0.0002 | 0.9560 $\\pm$ 0.0041 |\n| MLP-fix (transfer)     | 0.7311 $\\pm$ 0.0271 | 0.1617 $\\pm$ 0.0456 |\n\nThe local fidelity learned by transformer and MLP are essentially the same, but for global fidelity, transformer outperforms MLP. This reveals that **transformer is better at modeling the correlation in measurement data compared to MLP**. More importantly, transformer enables flexible transferability to quantum system of different sizes.\n\nFor architectural complexity, we highlight that our transformer encoder is relatively shallow. It has 2 blocks, each of which contains one 4-head MHA layer and the embedding dimension is only 128. Thus, the architecture is not complex compared with modern LLM. We just utilize its self-attention mechanism, a benefit for modeling quantum data as shown above. More implementation details can be found in the reply 2 to Reviewer cAXj.\n\n**3. Q2. Scalability to larger circuits.**\n\nIt's hard to guarantee the scalability to arbitrary large circuit with good global fidelity. Consider a simple case, where your goal is to learn a product state $\\bigotimes_{i=1}^n|\\psi_i\\rangle$. The agent learns to construct a circuit that maps this product state to $|0\\rangle^{\\otimes n}$ by maximizing the global fidelity, computed as $F=\\prod_i |\\langle\\psi_i|0\\rangle|^2$. Suppose the agent is good enough to fit every local term $|\\langle\\psi_i|0\\rangle|^2$ up to accuracy $1-\\epsilon$. The total accuracy of $F$ is $(1-\\epsilon)^n$, which decays exponentially with the system size. Intuitively, **the difficulty of maintaining good global fidelity would exponentially increase with the system size, even for learning such easy product states**. Nevertheless, in our experiment, we have shown that for learning states like Ising ground states, the performance of zero-shot transfer learning on 70 and 100 qubit systems **remains stable**, as shown in Figure 3(c) and Figure 4(b). This indicates that the agent can capture some underlying pattern of the quantum system that is transferable to another scale, thus we can expect a promising fidelity on relatively large system.\n\nWe further did experiments on learning 120 and 150 qubit Ising model. The following table records the results of local and global fidelity.\n\n| System size | Local               | Global              |\n| ----------- | ------------------- | ------------------- |\n| 120         | 0.9985 $\\pm$ 0.0006 | 0.9180 $\\pm$ 0.0333 |\n| 150         | 0.9984 $\\pm$ 0.0007 | 0.8927 $\\pm$ 0.0446 |\n\nThis shows that even though the fidelity drops with the increase of system size, it doesn't show an exponential degrade, validating the effectiveness of our model.\n\n\n**4. W1. Extension to more complex problems.**\n\nIn the following experiment, we consider learning 50-qubit random rotated maximally entangled states (GHZ), where the reduced density matrices are maximally mixed states. The agent has freedom to choose to apply Hadamard, CNOT and $R_z(\\theta)$ where $-\\pi / 4 < \\theta < \\pi / 4$. The table below shows the local and global fidelity between the reconstructed and target state.\n\n| Local               | Global              |\n| ------------------- | ------------------- |\n| 0.9927 $\\pm$ 0.0151 | 0.9275 $\\pm$ 0.1511 |\n\nWe can see that **although the states are maximally entangled, the agent can still successfully reconstruct them using the local fidelity reward function if choosing an appropriate action space**. This again highlights the flexibility and applicability of our framework.", "claims_and_evidence": "The claims the authors wish to explore are well validated. They demonstrate that their algorithm works and, under certain conditions, outperforms alternative approaches.", "methods_and_evaluation_criteria": "It would be more convincing if the authors discussed/demonstrated scaling to larger circuits or perhaps, more complex problems. However, the chosen benchmarks are relevant.", "theoretical_claims": "There are not many/any theoretical claims made in this manuscript.", "experimental_designs_or_analyses": "I had no issue with the experimental design or analysis.", "relation_to_broader_scientific_literature": "The approach to building quantum circuits using reinforcement learning is becoming more popular, so this paper aligns well with the emerging field. Further, using transformer architectures to encode entanglement is a novel concept.", "essential_references_not_discussed": "I am not familiar enough with the specific literature to discuss whether essential literature is complete. However, the authors do have a comprehensive state-of-the-art and related work section in both the main manuscript and the supplementary information.", "other_strengths_and_weaknesses": "The idea is novel and interesting to the broader community. A possible weakness is the extension to more complex problems but, possibly more importantly, larger circuits. Further, the claims regarding capturing of entanglement were not well validated. Would a simpler architecture have sufficed?", "other_comments_or_suggestions": "Some of the figures were quite small, 6 and 7 in particular.", "questions_for_authors": "1. Can you validate that the transformer architecture is a benefit here? Was there enough data to truly use this complexity and is there any way to explore the capturing of entanglement or correlation effects after training?\n2. Can you scale this arbitrarily to large circuit and expect good fidelity?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zi5SxNzzNu", "reviewer": "Reviewer_4FL5", "summary": "The paper introduces a novel approach termed \"explicit circuit representations\" for quantum state characterization. Unlike traditional implicit representations, this method allows for direct experimental reconstruction of quantum states. The representations are designed to predict quantum properties accurately based on local measurement data alone. A reinforcement learning-based framework called QCrep is developed to learn these explicit circuit representations. QCrep relies on a local fidelity-based reward function to train an agent, circumventing the barren plateau problem common in gradient-based quantum optimization. The framework uses a Transformer-based measurement feature aggregation block to capture global features of quantum states from local measurements.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Suggestion 1. Refinement of tables and plots.**\n\nTo improve readability, we will refine the tables and figures by moving the legends to the top or embedding them within the figures and abbreviating some metrics in tables. The figures in current paper is in PDF format thus can be scaled larger without loss of resolution.\n\n**2. Suggestion 2. Refinement of the Discussion section.**\n\nThe discussion section primarily focuses on evaluating the scalability of our method in more challenging scenarios. The experiments presented in this section shows potential challenges and solutions. We will refine this section by clarifying the topic of discussion and incorporating an exploration of the practical adaptation of our method.\n\n**3. Suggestion 3. Proof on mitigating BP.**\n\nWe would like to highlight that our proposed approach is not specifically targeting on mitigating BP, but state characterization (also not tomography). Our main focus is generating circuits for target family of states **locally**, which allows for computing the properties via measuring the output states, and for handling downstream tasks by directly decoding the learned representations without touching the underlying states.\n\n*Therefore, even though the result seems not surprising, it doesn't affect our contribution. In turn, the prior works have provided rigorous guarantee on on mitigating BP with local cost function, which empowers the effectiveness of our reward.* Local cost function has guaranteed trainability for larger systems [1–3]. However, **good trainability does not directly imply good global fidelity and the relation between local cost function and global fidelity remains unclear from the prior works**. Hence, we leverage the advantage of local cost function to design our reward for characterizing large scale quantum systems, and further show how global fidelity can be lower-bounded when the agent learns high local fidelity. Additionally, beyond the numerical experiment we present to demonstrate the mitigation of BP, our extraordinary performance in scaling to much larger systems (e.g., 50 qubits) compared to gradient-based methods also indirectly supports the mitigation of BP.\n\n[1] Sack et al. \"Avoiding barren plateaus using classical shadows.\" PRX Quantum 3.2 (2022): 020365.\n\n[2] Uvarov, A. V., and Jacob D. Biamonte. \"On barren plateaus and cost function locality in variational quantum algorithms.\" Journal of Physics A: Mathematical and Theoretical 54.24 (2021): 245301.\n\n[3] Cerezo, et al. \"Cost function dependent barren plateaus in shallow parametrized quantum circuits.\" Nature communications 12.1 (2021): 1791.\n\n**4. Q1. The advantage of transformer. The sample complexity of local measurement.**\n\nFor the advantage of transformer, please refer to the reply 2 to reviewer 9UgW.\n\nA key advantage of our framework over architecture search is that it requires no gradient computation with respect to the circuit parameters, thus can explore more regions during optimization. This feature ensures robustness against noise (see Appendix G), and is natural suitable for tensor network backend, which can simulation large-scale quantum systems. The back-prop from SVD layer is ill-defined when containing zero or repeated singular values if using gradient-based optimizer.\n\nRegarding sample complexity, please refer to Appendix F, and the reply 2 to Reviewer 5m67.\n\n**5. Q2. The generalization ability in the region of critical behaviors for many-body Hamiltonian task.**\n\nTo the best of our knowledge, accurately characterizing quantum states in critical region is an open problem of quantum state learning and is not the target of our approach. We use DMRG to simulate many-body ground states, which cannot approximate the real ground state well in the phase transition region. Nonetheless, our method could provide a means to detect phase transition by comparing the fidelity between the reconstructed state and the actual ground state.\n\n**6. Q3. Clarification on the comparison of Ising evolution.**\n\nYour understanding is absolutely correct. There's a misuse of notation of $t$ and we will correct this in a revised version. The trotter step is set to 0.1, as demonstrated in Appendix E. We compare the state fidelity between the learned state and first order trotter decomposed state as shown in Figure 3 (a) and Table 3, where the trotter decomposed state is can approximate the actual evolved state with error rate within 1e-6, but is less efficient if small error is allowed.\n\n**7. Q4. Learning quantum stabilizer codes.**\n\nThis is a very interesting topic. However, we note that for a QEC decoder, the output varies according to the input, which is not a quantum state characterization task. It is not compatible with our framework that maps a family of states towards a single state. Extending our framework to support quantum channel characterization could be important yet non-trivial during the short period of rebuttal and we would leave this for future work.", "claims_and_evidence": "1.The claim regarding local and global fidelity is well supported by rigorous proof.\n2.The effectiveness of the proposed representation is well supported by numerics.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes, i have checked the correctness of the proof in appendix, i.e. Proposition 2.1.", "experimental_designs_or_analyses": "I have checked all the experimental designs and they are the main way to validate their proposed framework.", "relation_to_broader_scientific_literature": "The application of reinforcement learning to the quantum problem (tomography) falls within the scoop of AI for Science.", "essential_references_not_discussed": "I think the related works are all currently disussed in the paper.", "other_strengths_and_weaknesses": "Strengths:\n1. The authors introduce local fidelities as reward function, resulting in polynomial sample complexity and claim to mitigate barren plateau.\n2. The authors bound the global fidelity well with averaged local fidelity, giving rise to a theoretical guarantee for the RL-based optimization.\n\nWeaknesses/Suggestions are listed below.", "other_comments_or_suggestions": "1. Obviously, the tables and plots are too small, and hard to read the content inside, which significantly affect the quality of this submission.\n2. The structure of the paper is not outlined well, e.g., the Discussion section reads like an extension to the former Experiment section.\n3. It is claimed that measuring local observables can mitigate BP; however, this is only supported by a single numerical experiment, with no rigorous proof quantifying the extent of its effectiveness. Moreover, the proposed mitigation approach appears to align with the method described in [1], making the result unsurprising.\n\n[1]. Cerezo, Marco, et al. \"Cost function dependent barren plateaus in shallow parametrized quantum circuits.\" Nature communications 12.1 (2021): 1791.", "questions_for_authors": "1. From what it is shown in Figure 1., QCrep framework is a strategy using RL-based method for choosing an appropriate ansatz for quantum state learning (or decoding) at each layer, it looks like an architecture search optimization problem, what’s the core advantage of using transformer-based layers in learning the local measurement data? What’s the sample complexity of the local measurement?\n2. When using many-body Hamiltonian for learning task, how is the generalization ability of QCrep in learning the circuit representations in the region of critical behaviors?\n3. Several confusions are raised in the comparison of ising model’s evolution. In section 3.2, the time-evolution parameter notation t overlaps the learning step t in the QCrep. And the authors said: “To exhibit the results, we average the performance on different parameters g for each evolution time t.”, is it meaning that QCrep treat the states with different ‘g’ and ‘t’ are notations ‘s’ from the set S, then the framework will learn the circuit representation for each ‘s’, resulting in storing V_{s,t} for them? What is the trotter step when showing first-order Trotter-Suzuki decomposition in Figure 3(b), and what’s the global fidelities of first-order trotter decomposition state? \n4. It is with particular interesting to learn a quantum stabilizer codes and their decoding process to respective logical codes. Can QCrep learn the decoding process efficiently? For example, take 5-qubit stabilizer code as training set, how many steps do QCrep need to approximate a perfect decoder?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The manuscript received four high-quality reviews and the authors provided rebuttals to all reviewers. Despite an intensive discussion, there is no agreement among the reviewers about the overall assessment: two reviewers argue for a weak reject while two argue for a plain accept. The different perspectives of the reviewers are interpreted by the AC as a sign that the topic of the manuscript is worthwhile being discussed in the whole community. Since none of the reviewers argues strongly to reject the paper and the overall grade is weak accept, the manuscript is accepted.
Decision: Accept (poster)