Learning Explicit Circuit Representations for Quantum States from Local Measurements

Authors: Manwen Liao, Yan Zhu, Weitian Zhang, Yuxiang Yang
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: quantum state learning, quantum circuit construction, local measurements, reinforcement learning

Abstract:
Characterizing quantum states is essential for advancing many quantum technologies. Recently, deep neural networks have been applied to learn quantum states by generating implicit representations that map them into classical vectors. Despite their success in predicting state properties, these representations remain a black box, lacking insights into strategies for experimental reconstruction. In this work, we aim to open this black box by developing explicit representations of quantum states through the generation of preparation circuits using a reinforcement learning agent with a local fidelity reward function. Relying solely on measurement data from a few neighboring qubits, our agent accurately recovers properties of target states. Specifically, we design a quantum measurement feature aggregation block which is used to extract global features of quantum states from local measurement data. We also provide a theoretical guarantee for the proposed local fidelity reward function. Extensive experiments demonstrate the effectiveness of our approach in learning various quantum states of up to 100 qubits, including those generated by Instantaneous Quantum Polynomial circuits, evolved by Ising Hamiltonians, and many-body ground states. The learned circuit representations can be further applied to Hamiltonian learning as a downstream task utilizing a simple linear model.

Decision: reject
Reviewer scores: 6, 3, 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1730384518843

thread_type:
official_review

<thread_object>
creator: reviewer_wrz5

<content>
summary:
This submission addresses quantum state tomography, proposing a reinforcement learning (RL)-based method to reconstruct a circuit that approximates an unknown target state by acting on the state |0>. The method leverages a reward function based on local fidelity and relies solely on two-body observables, which the authors claim yields polynomial sample complexity and avoids barren plateaus during optimization. Furthermore, the authors demonstrate that the reconstructed circuit enables additional insights, such as estimating Hamiltonian parameters when the target states are ground states from known Hamiltonian families (Hamiltonian Learning). Numerical experiments are provided for states generated by IQP circuits and time-evolved and ground states from Ising and Heisenberg Hamiltonians.

strengths:
-Applying RL to quantum state tomography is a promising and innovative approach, particularly in reconstructing a circuit that generates the target state.

-The use of direct circuit representation for state reconstruction represents a novel approach compared to existing methods cited in the manuscript.

weaknesses:
- The primary concerns with this paper are related to the soundness and clarity of key claims, especially around sample complexity and avoidance of barren plateaus:

Claims of Polynomial Sample Complexity: 
The paper asserts that the proposed method learns an unknown quantum state using only a polynomial number of measurements. However, this claim seems to contradict well-established lower bounds on the sample complexity required for full tomography. Specific points of concern are:

1)  The tomography task itself is not clearly defined, particularly regarding the distance measure used to quantify the accuracy of the reconstructed state. The authors mention minimizing a local fidelity but do not clarify whether this is the objective measure of the tomography task. Proposition 1 appears to imply that local fidelity serves as a proxy for global fidelity, which, if true, casts doubt on the claim of polynomial sample complexity.

2)  The claim of polynomial sample complexity is weakly supported. No theoretical motivation is provided. The only support comes from the empirical results where two-body measurements seem to work for the set of states considered in the task.

3) Proposition 1 states that if local fidelity is 1−ϵ, then global fidelity is at least 1−Nϵ for N qubits. However, this bound is only meaningful if ϵ scales as 1/poly(N). Otherwise, for large qubit numbers, the total fidelity bound becomes not meaningful. I think this issue appears also in the numerical results (Figure 2b), where, for example, the fidelity for 50 qubits improves only when the local fidelity error is very low. Moreover, even assuming the local-global fidelity bound holds, the authors provide no evidence that their algorithm can achieve the necessary local fidelity within polynomial measurements.

4) The numerical results, while promising for up to 50-qubit systems (or 100 when extrapolating the trained 50-qubit model), are not convincing.
The test cases involve states from a limited set of families that the authors claim can be efficiently represented using matrix product states (MPS). I suspect that the algorithm’s performance may stem from the specific structure of these states that makes them easy to learn.
Additionally, the algorithm’s search space appears to be adapted to the state family of interest, which simplifies the learning task and likely contributes to the favorable results. This tailored setup may overstate the algorithm's effectiveness on more general states.


Claims of avoidance of barren plateaus: 

5) Similar to the sample complexity claim, there is no theoretical support provided for the claim that the training algorithm avoids barren plateaus. Also no theoretical investigation is conducted on the time efficiency of the algorithm or the guarantees of not falling in a local minima.

questions:
Questions and Suggestions:

- Clearly define the target task. What specific input states are considered? What precision level is desired, and in which distance measure?
- Provide theoretical evidence to substantiate the claims regarding sample complexity and training time.
-  Could the authors test their algorithm on states that do not permit efficient MPS representations?
- What happens if the search space is not tailored to the specific family of states? Can the algorithm still perform well if a universal gate set is used instead?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730593200354

thread_type:
official_review

<thread_object>
creator: reviewer_vhhj

<content>
summary:
This paper proposes QCrep - a reinforcement learning framework for learning explicit quantum circuit representations from local measurements. The key novelty lies in generating interpretable circuit descriptions rather than implicit neural representations. The framework uses an attention-based feature aggregator to process local measurement data and a local fidelity reward function to guide the learning. The authors demonstrate QCrep's effectiveness on three families of quantum states (IQP circuits, Ising evolution, and many-body ground states) and show its utility for downstream tasks like Hamiltonian learning.

strengths:
1. The paper provides solid theoretical analysis, including:

- A proof relating local and global fidelity bounds (Property 1)
- Careful consideration of measurement setups and feature extraction
- Comprehensive empirical validation across multiple quantum state families

2. The framework addresses several key challenges:

- Achieves polynomial sample complexity through local measurements
- Demonstrates zero-shot transfer to different system sizes
- Shows robustness to finite sampling and circuit noise

3. The experimental evaluation shows that:

- Their approach outperforms existing methods (Classical Shadow, TQS, VQE, QAOA) across multiple metrics
- Their approach scales well to systems with many qubits

weaknesses:
1. The paper claims that their approach circumvents the barren plateaus problem by avoiding gradient-based optimization. However, it is well known that barren plateau problem still emerge when one utilizes gradient-free optimization. Barren plateau means the landscape is essentially flat, hence no local optimization will be able to solve the problem. It is also well known that local fidelity could partially circumvent the issue of barren plateau. And it looks like the authors avoid some barren plateau behavior by utilizing local fidelity. It is not clear how their proposed framework circumvents barren plateaus.

2. The reported numbers in the numerical experiments are very confusing. Spin-Z is just a sum of single-qubit Pauli Z operators. Hence, one would expect all the existing methods to perform very well for estimating Spin-Z. For example, in Table 2, it was shown that the error for existing methods are all at least 0.1, but their proposed method achieves an error of 2e-7. I could not find sufficient details in the submission about how these numbers are obtained. I am worried that the authors did not use the best practice for existing methods. Hence, the claimed improvement may not be sound.

3. I have similar concerns about the soundness of the reported numbers for existing methods for Table 3, 4, 5. For example, when classical shadow achieves a fidelity of 0.97, one would expect that the error for the other properties to be much smaller than 0.1, but the authors are reporting an error of around 4.

questions:
- Could the authors clarify what is their main innovation in circumventing barren plateau problems using either theoretical analysis or numerical experiments?

- Could the authors provide a very detailed description for all the numbers obtained in Table 2 (Evaluation results of learning states generated by 4-qubit IQP circuits)? 4-qubit IQP circuit is such a simple and restricted family of states that existing methods would perform much better than the numbers reported in the table.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730606949498

thread_type:
official_review

<thread_object>
creator: reviewer_m5ku

<content>
summary:
In this paper, the authors employ reinforcement learning techniques to learn a state circuit representation based on measurement data from neighboring qubits. They utilize a task-specific candidate gate pool as the action space and implement a local fidelity-based reward function, which is theoretically justified. Experimental results on Instantaneous Quantum Polynomial (IQP) circuits and many-body physics systems with up to 100 qubits suggest that their method achieves higher fidelities and lower estimation errors compared to classical shadow methods, TQS, VQE, and QAOA.

strengths:
1. The introduction of a local fidelity-based reward function mitigates the need for the costly computation of global fidelity.
2. The numerical results demonstrate the effectiveness of their method on large-scale systems.
3. The paper is well-presented.

weaknesses:
1. Utilizing the exact expectation value as the model input may not be practical.
2. The paper does not discuss resource consumption during training and inference, such as the number of required measurements, sample complexity, and training duration, which are critical for practical applications.
3. The action space is specifically designed to match the original state preparation circuits, which reduces the search space but limits applicability to more general states and scenarios where prior knowledge of the system is lacking.
4. Given that the reinforcement learning approach functions similarly to an architecture search tool, the experiments lack comparisons with other architecture search algorithms, such as QAS and DARTS.

questions:
1. Since each iteration of QCrep requires measurement data to calculate the reward function, could the demand for measurement resources limit its scalability to larger systems?
2. In the IQP circuit experiments, why are the single-qubit gate $Z[\alpha]$ restricted to two fixed gates? If $\alpha$ can take any value, how should the action space be defined?
3. In the experiments in Section 3.2, why is g chosen from [-2, -1] instead of considering both negative and positive values?
4. What system size is utilized in the experiments presented in Appendix F.1? There is particular interest in how the number of measurements affects performance across different system sizes.
5. For NISQ devices, how is the action space configured? Is it feasible to incorporate quantum channels into the action space?
6. Previous work has leveraged prior knowledge, such as geometric locality (which the states examined in this paper also possess), to enhance the learning of quantum states and system properties. What advantages does the proposed method offer compared to these approaches?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730724163319

thread_type:
official_review

<thread_object>
creator: reviewer_89hb

<content>
summary:
In this paper, the authors propose a reinforcement learning (RL) based adaptive circuit construction approach for quantum measurement related tasks such as state or process tomography. 
They aim to reconstruct any $n$-qubit pure state $|\psi\rangle$ by applying a unitary transformation $U$ to the computational basis state $|0\rangle^{\otimes n}$ and find the inverse unitary transformation.
Their method utilizes a reward function based on local fidelity and relies solely on two-body observables, which they claim results in polynomial sample complexity and avoids barren plateaus during optimization.
Numerical experiments of the proposed method are provided for learning of tasks upto $100$ qubits, specifically for states generated by Instantaneous Quantum Polynomial (IQP) circuits and time-evolved by many-body ground states of Ising and Heisenberg Hamiltonians.

strengths:
- The integration of RL and ML for adaptive circuit design in quantum state tomography represents a novel contribution to the field. In short, the method is a demonstration of AI for Science as it applies RL to solving a quantum problem.

- The authors develop an explicit circuit representation that facilitates the reconstruction of target states and the computation of relevant properties. This foundation enhances the applicability of their method to downstream tasks, such as Hamiltonian learning, thereby extending its utility in quantum computing.

weaknesses:
- The tomography task modeled within the RL framework is not clearly defined.
- Claims regarding polynomial sample complexity for learning an unknown state appear to contradict established lower bounds on sample complexity for full state tomography.
- The theoretical foundation (Proposition 1) suggesting that local fidelity can serve as a proxy for global fidelity has notable limitations:
  - This proposition holds primarily for area-law entangled states that are classically matrix product state (MPS) simulatable and may not apply to volume-law entangled states. As entanglement increases, local and global fidelity measures diverge, a limitation that should be explicitly discussed in the text.
  - Proposition 1 becomes vacuous for systems with large number of qubits, as it can be alluded from from Fig. 2. In the 4-qubit experiments, both local and global fidelities increase simultaneously. However, in the 50-qubit experiments, while local fidelity shows a consistent, monotonic increase, the global fidelity exhibits a sharp rise only at the end of the learning stage. The authors suggest that this abrupt increase in global fidelity is due to the presence of barren plateaus. This reasoning is somewhat unclear to me, as their initial motivation for minimizing local fidelity was based on the absence of barren plateaus.
  - If Proposition 1 is true then, it casts doubt on the claim of polynomial sample complexity. No theoretical or numerical evidence supporting this claim has been provided in the article.
- The authors have do not include most important baselines which are considered SOTA for the quantum state tomography problem: MPS QST and NNQS QSTs: restricted Boltzmann machines (RBM) and feed-forward neural networks with autoregressive architecture (ARN). Please have a look for relevant references within [1].
- Lack of open source code and reproducibility statement. 

[1] Kurmapu, Murali K., et al. "Reconstructing complex states of a 20-qubit quantum simulator." PRX Quantum 4.4 (2023)

questions:
- **Clarifications regarding the task**:
  - How are quantum circuits represented as RL states? What method does the RL agent use to select an action from this representation? For example, [2] employs one-hot encoding and tensor-based 3D encoding to represent circuits for neural network input. If such encodings are used then the authors should cite them as I think it is closely related to their work.
  - What is the specific metric being minimized, and to what level of precision is this metric evaluated for terminating the RL episode?
  - How are the parameters of the intermediate quantum circuits tuned or optimized during training? Are these parameters re-tuned when addressing downstream tasks?
  - Why is the action space (quantum gate set) customized for each task in your experiments? Can the authors present evidence that their method performs equivalently well when using a universal gate set $({RX(\theta), RY(\theta), RZ(\theta), CNOT})$ across all tasks?
  - The transformer-based measurement aggregator, which uses 2-local, nearest-neighbor Pauli measurements, lacks explanation. How do the authors derive local fidelity from these measurements? Do they estimate the average of local $|0\rangle\langle0|_i$ projections $Tr(O\rho)$ from the 2-local measurements? More detailed elaboration is required.

- Can the authors provide numerical or theoretical evidence supporting their claims regarding polynomial sample complexity in learning? At what stage do the authors start counting measurement samples—after the agent has been trained for a specific problem?

- Can the authors conduct experiments on state reconstruction within the QST problem for states that do not allow efficient MPS simulations (e.g., states with volume-law entanglement)?

- Can the authors provide numerical evidence to validate their claim of the existence of the barren plateaus in Fig. 2(b)?

- Why did the authors choose to compare their method only with SOTA baselines for the QST problem? Can they include comparisons with the methods highlighted in the Weaknesses section?

[2] Patel, Yash J., et al. "Curriculum reinforcement learning for quantum architecture search under hardware errors" ICLR (2024)
</content>
</thread_object>

</thread>
<thread>
creation date:
1730971918658

thread_type:
official_review

<thread_object>
creator: reviewer_tk92

<content>
summary:
The work focusses on learning circuits for characterizing quantum states. Explicit representation of quantum
states using circuits are provided. RL is used with a local fidelity reward function.Experiments are performed 
for quantum states up to 100 qubits.Results on ground state and Hamiltonian learning are presented.

strengths:
1. Local fidelity reward function proposed.
2. A theoretical guarantee for the learning function provided.
3. The ground states of the Ising and Heisenberg XYZ ground states are learnt well.

weaknesses:
1. Results are presented for shallow depth circuits. No indication is provided
on the increase of complexity with depth.
2. For the Hamiltonian learning, the circuits parameters are failed to be learnt from the 
linear model using VQE and QAOA. This puts some limitation on applicability.

questions:
1. Would some nonlinear model work for VQE and QAOA?
</content>
</thread_object>

</thread>


Metareview:
metareview:
The submission proposes a reinforcement learning (RL) framework for constructing explicit circuit representations of quantum states using local fidelity as a reward function. While the integration of RL and quantum state learning is innovative, the paper faced concerns from reviewers, including weak theoretical support for claims about polynomial sample complexity and barren plateau mitigation, the limited scope of numerical experiments (focused on MPS-representable states), and the lack of comparisons with key baselines.

Despite revisions and additional clarifications during the rebuttal process, these fundamental issues remain unresolved for most reviewers, leading to mixed evaluations. The numerical results demonstrate promise, especially for systems with up to 100 qubits, but their relevance is questioned due to the tailored setup and absence of guarantees for general applicability. The lack of open-source code and reproducibility further diminishes the work's impact.