Title: Hadamard Test is Sufficient for Efficient Quantum Gradient Estimation with Lie Algebraic Symmetries
Venue: NeurIPS 2025
Accepted: yes
Keywords: quantum machine learning

sample complexity

Lie algebras

shadow tomography
Abstract: Gradient estimation is a central challenge in training parameterized quantum circuits ( PQCs) for hybrid quantum-classical optimization and learning problems. This difficulty arises from several factors,  including the exponential dimensionality of the Hilbert spaces and the information loss in quantum measurements.  Existing estimators, such as finite difference and the parameter shift rule, often fail to adequately address these challenges for certain classes of PQCs. In this work, we propose a novel gradient estimation framework that leverages the underlying Lie algebraic structure of PQCs, combined with the Hadamard test. By analyzing the differential of the matrix exponential in Lie algebras, we derive an expression for the gradient as a linear combination of expectation values obtained via Hadamard tests. The coefficients in this decomposition depend solely on the circuit's parameterization and can be computed efficiently. Also, these expectation values can be estimated using state-of-the-art shadow tomography techniques. Our approach enables efficient gradient estimation, requiring a number of measurement shots that scales logarithmically with the number of parameters, and with polynomial classical and quantum time. This is an exponential reduction in the measurement cost and a polynomial speed-up in time compared to existing works.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an efficient framework for gradient estimation in parameterized quantum circuits (PQCs) used in variational quantum algorithms (VQAs). While traditional methods such as finite differences and the parameter shift rule (PSR) suffer from inefficiencies and limited applicability, the proposed approach overcomes these limitations by leveraging the Lie algebraic structure of PQCs along with the Hadamard test. To achieve this, the authors derive an analytical expression for the gradient of a generic PQC using the differential of the matrix exponential map in Lie algebras. This expression represents the gradient as a linear combination of expectation values computable via Hadamard tests under certain constraints. Moreover, these expectation values can be efficiently estimated using shadow tomography, resulting in a logarithmic scaling of the number of measurement shots when the shadow norms of the observables are bounded, with the number of parameters and polynomial time complexity in both classical and quantum resources.\n\nSTRENGTHS AND WEAKNESSES:\nFrom my perspective, this is a technically solid work providing a gradient estimation algorithm for VQAs with provably fewer requirements on the quantum resources in some realistic settings. Although I feel that similar ideas of encoding Pauli strings into binary representations have been proposed, the observation that this encoding enables efficient characterization of their commutation relations and works particularly well when the set of Pauli terms in the Hamiltonian is closed under commutation is essential. This observation further allows a finite and efficiently computable gradient representation. Even when this closure property does not hold, the paper proposes approximation strategies via truncation or Poisson sampling. Overall, I believe this work should be a natural fit for NeurIPS 2025. \n\nRegarding the weakness, I only have some minor comments that do not affect my overall evaluation. \n\n- I suggest the authors using the notation $n$ instead of $d$ to represent the system size (number of qubits). I think $d=2^n$ usually refers to the dimension of the Hilbert space. I notice that the authors also use $n$ for system size at line 103, maybe due to a typo.\n\n- I suggest the authors mentioning a recent work [Abbas et al, NeurIPS 2023]. In this work, a backpropagation algorithm for VQAs is provided based on another line of shadow tomography techniques (Actually, I think the shadow tomography technique in this submission should be called classical shadow). A comparison between this submission and the recent work mentioned should be helpful.\n\n- There is a typo in the equation below line 175: $b^2$ should be $b^1$.\n\nQUESTIONS:\nI would like the authors to fix the minor points in the Strengths And Weaknesses part.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new framework for performing quantum gradient estimation in hybrid quantum-classical setups such as VQAs. The paper crucially identifies that the presence of symmetries in the Lie algebraic structure can exponentially reduce the sample complexity and number of measurements, while conceding at most a polynomial speedup during the classical post processing phase, in comparison to existing PSR based techniques in the literature. This uniquely positions the contributions of the paper as complementary to the existing literature on gradient estimation in hybrid setups.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n----\n\nThe paper is extremely well written. The assumptions are reasonable, well studied in existing literature, and clearly stated. The theorem statements and the proofs are easy to read. The contributions are well placed in the context of existing results such as [WLW+24]. The Lie-algebraic Hadamard-test framework is of broad interest in the community in its own right.\n\nWeaknesses\n---\n\nIt has been known for some time that constraining the underlying Lie-algebraic structure paves the way for classical simultability results, for example see [GLC+23]. The paper does not do a good job of convincing the reader why one would still prefer to have hybrid frameworks for gradient estimation, given the cost of quantum resources involved. See the questions section for this one point of weakness that is not properly addressed.\n\nIn my opinion, the paper is devoid of any major weaknesses barring its lack of comparison with respect to classical simulatibility results.  Another minor squabble one might have is the lack of experimental results demonstrating that the proposed framework performs well in practice. Since I am a theorist myself, I lack the technical know-how to suggest incorporating reasonable experiments into the paper, and I will defer to the opinion of the other reviewers on this matter. However,  in my opinion, the theoretical contributions of this work are solid enough to merit a discussion for acceptance.\n\nThere are a few minor typos I list here:\n\n- Line 27: Extra \"the\" before convergence rates.\n- Line 30: The phrase \"each gradient can have an exponential sample complexity\" does not make sense. I assume the authors are referring to the sample complexity of **gradient estimation**.\n- Line 146: Extra \"the\" before phase estimation.\n- Line 151: Missing \"the\" before Hadamard test.\n- Line 481-482: There should be $D_i$s instead of $D_i$ in multiple places.\n\nQUESTIONS:\nIt is unclear to me when the framework proposed in this paper would be more beneficial in the context of the classical simulatibility results of [GLC+23]. If the observable $O$ does not have polynomial dimensionality in the number of qubits, does the shadow norm of $O$ still remain bounded? How do we compare the sample complexity and time complexity of the current paper with those of [GLC+23] and other classical simulatibility techniques?\n\n\n----\n\n**Note to authors:** I am extremely willing to improve my score if the authors provide a reasonable justification to the above question.\n\nLIMITATIONS:\nSee above sections - weaknesses and questions for the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of gradient estimation in parameterized quantum circuits (PQCs). Specifically, it investigates an approach based on Lie algebraic structures within PQCs. Without loss of generality, a PQC can be represented as the exponential of $iA$, where $A$ is a parameterized Hamiltonian expressed as a linear combination of Pauli strings. In this formulation, the parameters of the PQC correspond to the coefficients of these Pauli strings.\n\nThe algorithm begins with the observation that the Hadamard test can be used to estimate both the real and imaginary components of $\\langle\\psi|U|\\psi\\rangle$ for a given unitary $U$ and quantum state $\\ket{\\psi}$. This procedure can be extended to estimate the quantity $D = \\mathrm{Tr}\\left(O[\\sigma, \\rho]\\right)$, where $O$ is an observable, $\\sigma$ is a Pauli string, and $\\rho$ is the output state of the quantum circuit. The authors show that, in the special case where the Pauli decomposition of the Hamiltonian $A$ is closed under commutation, the gradient of the PQC can be computed using $p$ distinct values of $D$, each associated with a different Pauli string, requiring $p$ corresponding Hadamard tests. Furthermore, the number of measurement shots can be reduced to $O(\\log p)$ by leveraging techniques from shadow tomography. To address the more general case where the Pauli decomposition is not closed under commutation, the authors propose three solutions: computing the closure, applying truncation, and using Poisson sampling.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The algorithm and the corresponding analysis is novel and interesting.\n\n2. The algorithm obtains a significant improvement on the number of shots in the special case where the decomposition is closed under commutation.\n\n\nWeaknesses:\n\n1. Beyond the examples provided in the paper, it remains unclear how broadly applicable the proposed framework is to more general settings. Please refer to the \"Questions\" part for more details\n2. The organization and presentation of part of the results are not super clear. Please refer to the \"Questions\" part for more details.\n\nQUESTIONS:\n1. Could you elaborate more on why WLOG we can assume the parameters of the PQCs are the coefficients of the Pauli decomposition? If we use different parameter choices for the same circuit family, would your results still apply?\n\n2. In Section 3.3, can you give a self-contained theorem statement on the overall resources for gradient estimation, similar to Theorem 2?\n\n3. In Section 5, can you give some self-contained description on the overall resources for gradient estimation for the three solutions you mentioned, similar to Theorem 2?\n\n4. Can you elaborate more on how Eq. 3 is obtained? I think it would be beneficial for readers to better understand the overall algorithm\n\n5. Typos & minor issues: Below line 175 it should be b^1 instead of b^2; in Eq. 6 it is probably better not to use 'i' as subscript given that you already used it to denote the imaginary unit;\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an efficient method for estimating gradients in parameterized quantum circuits (PQCs) by utilizing Lie algebraic structures in combination with the Hadamard test. Traditional methods often face exponential complexity or computational overhead, but the authors derive a concise mathematical formulation leveraging commutation relations, significantly reducing the computational complexity. Their method integrates classical shadow tomography, achieving logarithmic sample complexity in the number of parameters, and polynomial classical and quantum runtime, thus substantially improving efficiency for quantum optimization and learning tasks.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper does a nice job introducing a theoretically rigorous method for efficient gradient estimation in parameterized quantum circuits, using Lie algebra and Hadamard tests to significantly cut down complexity. Its theoretical analysis is clear and thorough, highlighting substantial improvements in efficiency. However, a major downside is the limited generality—the key condition (commutators forming a closed algebra) means it won't work for highly expressive or universal quantum circuits. Given this limitation, it would have been helpful if the authors provided more concrete examples of realistic quantum systems to clearly illustrate practical usefulness.\n\nQUESTIONS:\n1. Could you discuss explicitly how restrictive your condition (closure of commutators) is in practice? It would be helpful to provide more intuition or realistic examples of quantum systems that do or do not satisfy this condition.\n \n\n2. Given that your theoretical results have restricted generality, can you provide additional examples or numerical experiments involving realistic quantum applications (e.g., quantum chemistry Hamiltonians, QAOA circuits, or real-world quantum hardware ansätze)?\n \n\n3. Your method achieves impressive complexity improvements but sacrifices generality. Could you discuss or compare explicitly this trade-off against existing gradient estimation methods, especially highlighting scenarios where your method significantly outperforms others despite its restrictions?\n \n\n4. Are there natural or straightforward extensions of your framework to handle circuits whose commutators do not form a closed group, possibly via approximations or truncation? If so, briefly outline how that might work.\n \n\n5. Could you clarify the practical feasibility and computational overhead of employing shadow tomography in realistic scenarios? Specifically, under which conditions would shadow tomography actually offer substantial practical benefits over standard measurement techniques?\n\nLIMITATIONS:\nThe authors clearly acknowledge the primary limitation of their work—the reliance on a closed Lie algebra structure with polynomial dimensionality—which restricts the method’s applicability to non-universal quantum circuits. They also discuss strategies to address this, such as closure computation, truncation, and Poisson sampling. While there is no explicit discussion of societal impact, the work is theoretical in nature and focused on quantum algorithm optimization, which does not currently pose immediate societal risks. No critical omissions were identified in this regard.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-16"}