SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Amira Abbas
Google Quantum AI,Venice,California 90291,USA
University of KwaZulu-Natal, South Africa
QuSoft, University of Amsterdam, Science Park 123, 1098 XG Amsterdam, The Netherlands
Robbie King
Department of Computing and Mathematical Sciences, Caltech, Pasadena, CA 91125, USA
Hsin-Yuan Huang
Institute for Quantum Information and Matter, Caltech, Pasadena, CA 91125, USA
William J. Huggins
Google Quantum AI,Venice, California 90291,USA
Ramis Movassagh
Dar Gilboa
Jarrod R. McClean
jmcclean@google.com
The success of modern deep learning hinges on the ability to train neural networks at scale. Through clever reuse of intermediate information, backpropagation facilitates training through gradient computation at a total cost roughly proportional to running the function, rather than incurring an additional factor proportional to the number of parameters – which can now be in the trillions. Naively, one expects that quantum measurement collapse entirely rules out the reuse of quantum information as in backpropagation. But recent developments in shadow tomography, which assumes access to multiple copies of a quantum state, have challenged that notion. Here, we investigate whether parameterized quantum models can train as efficiently as classical neural networks. We show that achieving backpropagation scaling is impossible without access to multiple copies of a state. With this added ability, we introduce an algorithm with foundations in shadow tomography that matches backpropagation scaling in quantum resources while reducing classical auxiliary computational costs to open problems in shadow tomography. These results highlight the nuance of reusing quantum information for practical purposes and clarify the unique difficulties in training large quantum models, which could alter the course of quantum machine learning.
Figure 1: Quantum backpropagation algorithm. Our proposal for quantum backpropagation consists of an online shadow tomography protocol, coupled with a threshold search procedure [Aaronson et al., 2018, Bădescu and O'Donnell, 2021]. The algorithm is executed in batches of size  $O(\mathrm{polylog}(M))$ , of which roughly  $n$  batches are needed, where  $\rho$  is an  $n$  qubit quantum state. A classically constructed hypothesis state  $\sigma$  is also necessary for the algorithm. Crucially, quantum states and the hypothesis state are rotated before each threshold check, to rotate through the layers of a quantum neural network  $F(\theta)$ ,  $\theta \in \mathbb{R}^{M}$  and reuse information for gradients. This enables a cost reduction from  $O(M^2 \cdot \mathrm{polylog}(M))$  to  $O(M \cdot \mathrm{polylog}(M))$  to compute the full gradient. For convenience, we suppress precision factors which scale as  $O(1 / \varepsilon^4)$  for this proposal.
Computing gradients through backpropagation is crucial to the success of modern deep neural networks. Rather than a naive manifestation of the chain rule to compute gradients, backpropagation leverages white-box knowledge of a computational graph, as well as intermediate values, to asymptotically improve run times [Goodfellow et al., 2016, Rumelhart et al., 1986, LeCun et al., 2015, 1989, Bengio et al., 2014]. Remarkably, computing the gradient of, say, a neural network function with respect to all its parameters, can be done at a total cost roughly proportional to running the function, instead of incurring an additional factor proportional to the number of parameters. This relative scaling, owed to backpropagation, has facilitated the training of very deep networks, with parameter counts now in order of  $10^{10}$  - accompanied with unparalleled empirical success [Goodfellow et al., 2016, Szegedy et al., 2017, Iandola et al., 2016, Wu et al., 2019]. When considering the number of function calls required to compute gradients, backpropagation in classical circuits remains exponentially more efficient with respect to the number of parameters, than the best known algorithms for determining gradients of parameterized quantum circuits - with or without the aid of a quantum computer [Gilyén et al., 2019, Van Apeldonorn et al., 2020, Brandão et al., 2017, Jordan, 2005, Schuld and Killoran, 2022]. Nevertheless, the allure of large-scale models inspires the need for efficient training of parameterized quantum models [Kandala et al., 2017, Farhi et al., 2014, Cerezo et al., 2021], which frequently arise in fields like quantum machine learning and quantum chemistry. But if backpropagation scaling cannot be matched, practically reaching overparameterized regimes may be impossible, even before accounting for additional challenges like barren plateaus [McClean et al., 2018, Wang et al., 2021, Cerezo et al., 2020]. Since trainability influences the power and applicability of a model, this could radically shift the current trajectory of preferred quantum models.
In this work, we provide an operational definition of backpropagation, and subsequently determine its feasibility for parameterized quantum models. We investigate learning algorithms with and without quantum memory, where the former is able to store a product of multiple copies of a particular state, perform joint quantum operations followed by an entangled measurement. Whereas, a learning algorithm without quantum memory can only perform operations on each copy, implement a (conditional) measurement, and use the resulting classical data. Without access to multiple copies, we highlight that all known methods to compute gradients of simple variational models, do not achieve an overhead in line with backpropagation, unless one considers very special cases. Interestingly, closely related probabilistic classical analogues can exhibit backpropagation scaling, which points out that the barrier in the quantum setting is due to quantum phenomena. In an attempt to mimic classical backpropagation, which leverages information reuse to produce a favourable scaling, we lean on a similar concept in a quantum setting, namely gentle measurements [Aaronson, 2019, Aaronson and Rothblum, 2019, Bădescu and O'Donnell, 2021]. When combined with online learning [Aaronson
et al., 2018], this technique has proven useful in problems like shadow tomography as it aims to conserve quantum resources, but has yet to be explored in the context of backpropagation. In an information-theoretic sense, when access to multiple copies of a state is provided, a modification of existing shadow tomography routines enables backpropagation scaling if one restricts costs to the quantum overhead and ignores the classical cost incurred to implement known shadow tomography schemes. We present our proposed quantum backpropagation algorithm in Figure 1 which highlights the reduction in quantum resources due to the ability to exploit structure in a quantum neural network and reuse information through gentle measurement. Unfortunately, the true computational efficiency of our scheme remains an open question and we rule out a general strategy based on gentle measurement alone, by linking to computational models known to be more powerful than those contained in the complexity class of BQP [Aaronson et al., 2016]. Despite failure to achieve backpropagation scaling in this general setting, the construction is suggestive of approximate or restricted models that may yield the desired scaling without violating complexity-theoretic bounds. This avenue remains rich for future work. We hope that these results illustrate the difficulty of replicating backpropagation scaling in parameterized quantum circuits and inspire the development of alternative quantum models that can train at scale.
2 Backpropagation scaling
A comprehensive overview of gradient evaluation, given by automatic differentiation on classical computers, can be found in Griewank and Walther [2008]. The key advantage, however, can be summarized in one sentence: computational and memory resources employed to compute gradients of a function are bounded multiples of those used to compute the function. We use this bound to define the requirements for backpropagation scaling.
Definition 1 (Backpropagation scaling). Given a parameterized function  $F(\theta)$ ,  $\theta \in \mathbb{R}^M$ , let  $F'(\theta)$  be an estimate of the gradient vector accurate to within some constant  $\varepsilon$  in the infinity norm. The total computational cost incurred to obtain  $F'(\theta)$  with backpropagation is bounded such that
$$
\operatorname {T I M E} \left(F ^ {\prime} (\theta)\right) \leq c _ {t} \cdot \operatorname {T I M E} (F (\theta)), \tag {1}
$$
$$
\operatorname {M E M O R Y} \left(F ^ {\prime} (\theta)\right) \leq c _ {m} \cdot \operatorname {M E M O R Y} (F (\theta)), \tag {2}
$$
where  $c_{t}, c_{m} = \mathrm{polylog}(M)$ , and  $\mathrm{TIME}(\cdot)$  and  $\mathrm{MEMORY}(\cdot)$  capture the time and space complexity respectively, for either computing the function  $F$  or its gradient  $F'$ .
As a further specification of backpropagation scaling in Definition 1, one can specify whether one achieves this scaling in quantum resources, classical resources, or all resources. While it is, of course, the goal to achieve this scaling in all resources, the distinction remains relevant due to the ability to leverage classical resources in order to improve the scaling in quantum resources, which we elaborate on in Section 5. In purely classical models, like neural networks, the overhead for both time and memory can be constant, and typically by a small factor. This efficiency has been instrumental for training very large models and is arguably the main contributor to the success of modern day machine learning. Given that variational quantum models, which utilize parameterized quantum circuits, are believed to be the most promising candidates to solve quantum machine learning tasks, we investigate their ability to reproduce this scaling.
3 Variational quantum models
Variational algorithms have become a go-to approach when looking to solve various optimization and machine learning problems on quantum devices [Cerezo et al., 2021]. We present a slightly restricted model for ease of analysis which still covers a very broad range of practical scenarios. Notably, if backpropagation scaling cannot be achieved in this simplified setting, it is unlikely to succeed in a more sophisticated one.
Definition 2 (Simple variational model). Consider an initial quantum state  $\rho$  and a quantum circuit with  $M$  parameterized operations  $U_{j}(\theta_{j}) = e^{-i\theta_{j}P_{j}}$ , where each  $P_{j}$  is a Pauli operator acting on up to  $n$  qubits. We define a simple variational quantum model as the parameterized function
$$
F (\theta) = \operatorname {T r} [ O \rho (\theta) ], \tag {3}
$$
where  $O$  is a Hermitian and unitary observable, and the quantum state  $\rho(\theta)$  is expressed as  $\rho(\theta) = U(\theta)\rho U(\theta)^{\dagger}$ . In the most general case,  $\rho$  will be an unknown quantum state that we refer to as the quantum data setting, but we will also be interested in the simplified setting where  $\rho(\theta) = |\psi(\theta)\rangle\langle\psi(\theta)|$  and  $|\psi(\theta)\rangle = U(\theta)|0\rangle^{\otimes n} = \prod_{j=1}^{M}U_j(\theta_j)|0\rangle^{\otimes n}$ .
In the simplified setting, the  $k^{\mathrm{th}}$  gradient component of  $F(\theta)$  can be expressed as
$$
\left[ F ^ {\prime} (\theta) \right] _ {\theta_ {k}} = 2 \operatorname {R e} \left[ \langle 0 | \left(\prod_ {j = 1} ^ {M} e ^ {i \theta_ {j} P _ {j}}\right) O \left(\prod_ {m = k + 1} ^ {M} e ^ {- i \theta_ {m} P _ {m}}\right) (- i P _ {k}) \left(\prod_ {l = 1} ^ {k} e ^ {- i \theta_ {l} P _ {l}}\right) | 0 \rangle \right]. \tag {4}
$$
It becomes clear that computing all  $M$  components involves a large number of common operations. At face value, one might think it straightforward to exploit this overlap of operations to gain computational efficiency, as is done classically. The problem is that intermediate information in a quantum circuit is not easily retrievable without consequence, which is investigated in the following section.
4 Learning algorithms without quantum memory
Recall that a learning algorithm without quantum memory would perform operations and measurements on each individual copy of a quantum state. In this regime, which is prominent in current quantum machine learning settings, we have the following proposition.
Proposition 3 (Backpropagation scaling is impossible for quantum data using single copies). Given the quantum data setting where one seeks to train a variational model using copies of the unknown state  $\rho$  and the additional constraint of no quantum memory, then backpropagation scaling is not possible in the general case.
Proof. Take the Pauli circuit model above and let us consider the case of all possible Pauli operators  $P_{j}$  on  $n$  qubits, such that  $M = O(4^{n})$ . If we take the special case of quantum data and initializing all  $\theta_{j} = 0$ , then the gradient with respect to each of the parameters is given by the expected value of all possible Pauli operators on  $n$  qubits on the unknown quantum state  $\rho$ , up to a small constant. If no quantum memory is available, that is, we only have the ability to perform measurements on single copies at a time, then by Chen et al. [2022, Corollary 5.9], the minimal number of copies of  $\rho$  is lower bounded by  $\Omega (2^n /\varepsilon^2)$  in order to predict all Pauli operators to at most  $\varepsilon$ -error with probability  $2 / 3$ . Hence, backpropagation scaling is not possible in general in the single copy case.
Notably, Proposition 3 is based on an information-theoretic separation that does not generalize to the simplified case,  $\rho = |0\rangle \langle 0|$ , or even when  $\rho$  is simply guaranteed to be a pure state generated by a polynomial sized circuit, which we detail in Appendix C. Hence, for the simplified case and polynomial complexity pure state cases, we must turn to computational arguments. Furthermore, if it were possible to find a polynomial time algorithm for the approach in Appendix C, then it would be possible to efficiently clone pseudo-random states, which is not believed to be possible [Ji et al., 2018], despite the fact that they are pure states generated by polynomial sized circuits (see Appendix C.2). The following remark aims to clarify the status of current methods for approaching this problem.
Remark 4 (Current gradient methods fail to achieve backpropagation scaling). Given a variational model  $F(\theta)$  defined in (3) with time complexity
$$
\operatorname {T I M E} (F (\theta)) = \tilde {O} (M / \varepsilon^ {k}),
$$
for some integer  $k$  and precision  $\varepsilon$ , then all known schemes to estimate the gradient of  $F(\theta)$  to the same precision, do not, in general, achieve a time complexity in line with backpropagation scaling.
We briefly explain why known gradient methods fail, but defer details to Appendix A. A promising gradient algorithm put forth in Jordan [2005] requires only a single black-box query to a function to estimate its full gradient with a desired precision. But, as shown in Gilyén et al. [2019], when considering variational models, a different query model must be applied and the original single-query advantage becomes unattainable. The authors derive lower bounds requiring a quantum computational cost of  $O(M\sqrt{M} / \varepsilon^2)$  and in a high precision regime,  $O(M\sqrt{M} / \varepsilon)$  is worst-case
optimal [Huggins et al., 2021] when using a black-box simplified model of  $U(\theta)$ . In other contexts, it is also sometimes argued that the simultaneous perturbation stochastic approximation (SPSA) algorithm is computationally efficient since it requires two function evaluations to estimate the gradient, irrespective of  $M$ . This seemingly satisfies the scaling required, however, as  $M$  increases, the variance of the gradient estimate increases and, thus, to counteract this, either a smaller learning rate must be used - increasing the number of optimization steps - or more samples are needed to estimate the gradient with an appropriate accuracy at every step. We derive a sample complexity bound in Appendix A.2.3 which demonstrates SPSA's inability to exhibit backpropagation scaling. Thus far, other sampling schemes constructed to estimate the gradient of  $F(\theta)$ , like the parameter-shift rule, perform destructive measurements that typically only retrieve a partial amount of information for one component of the gradient. As a result, reducing the infinity norm error in the gradient with reasonable probability, has a cost that scales like converging each component, i.e.
$$
\operatorname {T I M E} \left(F ^ {\prime} (\theta)\right) \propto M \operatorname {T I M E} (F (\theta)) \tag {5}
$$
$$
= \tilde {O} \left(M ^ {2} / \varepsilon^ {k}\right), \tag {6}
$$
which unfortunately, does not achieve backpropagation scaling.
While this quadratic dependence on the number of parameters may not seem problematic, a linear dependence was the necessary catalyst in the age of modern deep learning, with overparameterized networks that perform exceedingly well on practical tasks. We illustrate the consequences of quadratic scaling in Appendix A.2, Figure 2, where one could wait up to a day to evaluate a single gradient estimate of a model with fewer than 10 000 parameters.
But perhaps neural networks are not a fair benchmark. One could dig deeper in automatic differentiation literature to investigate whether a direct classical analogue for these parameterized quantum circuits attains backpropagation scaling. Interestingly, a particular analogue can.
Proposition 5 (Classical analogue achieves backpropagation scaling). Parameterized Markov chains, which are much closer classical analogues to variational models than neural networks, exhibit backpropagation scaling.
We detail the proof and scaling comparison in Appendix B by drawing an analogy between quantum and classical probabilistic states. Under some reasonable assumptions on the set of classical operations, the desired scaling is indeed possible in analogous classical parameterized stochastic processes. The formulation of this classical-quantum analogy allows us to probe the root cause of why backpropagation scaling is so difficult to obtain in the quantum variational setting. The origin of the challenge lies within quantum measurement collapse and the inability to read out intermediate states while continuing a computation, rather than the probabilistic formulation of the problem. In the classical setting, one is always promised to be in a computational basis state, making it possible to do perfect measurements non-destructively at intermediate steps. It remains an interesting open question to better understand the performance separation on practical tasks between quantum variational methods and this type of classical analogue, given the advantage in trainability of the latter.
Although Proposition 3 presents a strict lower bound ruling out backpropagation in the quantum data case with single copies, this leads one to wonder whether backpropagation scaling is possible when one has access to multiple copies. Moreover, destructive quantum measurements are the inhibitor of backpropagation scaling in single copies, so perhaps there is some middle ground where one could perform measurements that are only partially destructive on multiple copies. This idea has led to breakthroughs in the shadow tomography problem, which we examine next in the context of backpropagation.
5 Reusing multiple copies through gentle measurement
By allowing access to multiple copies of  $\rho$ , it is especially interesting to note that gentle measurements can facilitate backpropagation scaling in all resources, when considering the special case outlined in Proposition 3. We first define gentle measurement, followed by the special case construction.
Definition 6 (Gentle measurement). Fix a subset of quantum mixed states  $S$ . A measurement  $F$  is  $\alpha$ -gentle on  $S$  if for every state  $\rho \in S$ , and every outcome  $y$  of  $F$ , the post-selected state  $\rho_{F=y}$  obeys
$$
\left| \left| \rho_ {F = y} - \rho \right| \right| \leq \alpha ,
$$
where  $\alpha \in [0,1]$ . Hence, the smaller the  $\alpha$ , the less damage incurred by  $\rho$ .
Proposition 7 (A special case variational model achieves backpropagation scaling). Given a variational model  $F(\theta) = \mathrm{tr}\big[U(\theta)\rho U(\theta)^{\dagger}\big]$ , where  $U(\theta) = \prod_{j=1}^{M} e^{-i\theta_j P_j} V$  for some fixed unitary  $V$ , setting  $\theta$  to zero and  $O = I$ , the  $k^{th}$  gradient component may be written as
$$
F ^ {\prime} (0) _ {k} = 2 \operatorname {R e} \left(\operatorname {T r} \left[ V \rho V ^ {\dagger} (- i) P _ {k} \right]\right) = 2 \operatorname {I m} \left(\operatorname {T r} \left[ V \rho V ^ {\dagger} P _ {k} \right]\right). \tag {7}
$$
Then all  $M$  gradient components can be estimated to within a fixed precision  $\varepsilon$  using  $O(\log (M) / \varepsilon^4)$  function calls, and thus,
$$
\operatorname {T I M E} \left(F ^ {\prime} (\theta)\right) = O (\log (M)) \operatorname {T I M E} (F (\theta)),
$$
which is in line with backpropagation scaling.
Proof. With use of an ancilla qubit and a slightly modified circuit, the imaginary components of  $\mathrm{Tr}[V\rho V^{\dagger}P_k]$ , which represent gradient components, can be estimated with  $O(\log M / \varepsilon^{4})$  copies of  $\rho$  for all  $k$  via application of  $\mathbf{V}$  and two-copy Bell measurements of the resulting state that harness gentleness. Similarly, using  $O(\log (M) / \varepsilon^{2})$  copies along with the magnitude information from the Bell measurements, one may estimate the sign of the gradient components using a majority vote scheme that also exploits gentleness. The total number of copies scales as  $O(\log (M) / \varepsilon^{4})$ , inducing a time complexity of  $O(\log (M)\cdot M / \varepsilon^{4})$  with efficient classical overhead. The details of the implementation are discussed in Huang et al. [2021, Appendix E].
This result indicates that there are at least some choices of circuits and generators for which backpropagation scaling can be achieved using gentle measurements. The exception naturally leads one to ask if this may be possible in more general cases with techniques like shadow tomography [Aaronson, 2019], however, with just a small perturbation away from this special case, the same technique no longer works, and the general computational efficiency remains unknown [Aaronson and Rothblum, 2019, Bădescu and O'Donnell, 2021]. In the subsequent section, we adapt shadow tomography results and exploit the sequential structure in variational models equipped with quantum data, to obtain backpropagation scaling in quantum resources, but leave open the question of classical computational efficiency. This represents substantial progress over current gradient methods for these models.
5.1 A quantum-efficient protocol for backpropagation