SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where we use  $p(i_1, \dots, i_N)$  to represent the probability of a particular configuration that was sampled, and similarly  $O(i_N)$  for the value of the final configuration. We assume that for each individual configuration it is possible to compute the transition probability between individual configurations, e.g.  $(i_N | U_N | i_{N-1})$  which is typically true in the classical case as well. As a result, for a given path, we use re-weighting to make that path produce an unbiased sample for the gradient component we are interested in as well. In particular, writing the same for the shifted gradient estimator for component  $j$  merely requires substituting the relevant matrix element
$$
\left(i _ {j} \mid U _ {j} \mid i _ {j - 1}\right)\rightarrow \left(i _ {j} \mid H _ {j} U _ {j} \mid i _ {j - 1}\right) \tag {29}
$$
hence we can estimate the gradient using samples re-weighted by
$$
\partial_ {\theta_ {j}} f (\theta) = \sum_ {i _ {1}, \dots , i _ {N}} p \left(i _ {1}, \dots , i _ {N}\right) \left(\frac {\left(i _ {j} \mid H _ {j} U _ {j} \mid i _ {j - 1}\right)}{\left(i _ {j} \mid U _ {j} \mid i _ {j - 1}\right)}\right) O \left(i _ {N}\right) \tag {30}
$$
where the weighting factors we also assume to be efficiently computable by construction of the elementary operations  $U_{i}$ , which is analogous to the quantum generators typically used as well, defined as simple operations lifted into large spaces by tensor products. This suggests the following procedure for efficiently estimating gradients with respect to parameters in the classical analog of quantum variational circuits.
1. Draw a sample from  $\psi^0$  and store this configuration as  $|i_i)$ , which may be represented efficiently as a classical bit string.
2. For each elementary operation  $U_{i}$ , sample the next classical configuration with probability determined by  $U_{i}$ , and store the configuration as  $|i_j\rangle$ .
3. Upon reaching the final configuration, evaluate  $O(i_{N})$  from the definition of  $O$  to determine the value of the objective.
4. Using the stored path,  $\{|i_j)\}$ , for each elementary step, sample  $\left(\frac{(i_j|H_jU_j|i_{j-1})}{(i_j|U_j|i_{j-1})}\right)O(i_N)$  and store the value in a vector to be used in a running average that determines the gradient.
5. Repeat this procedure until the uncertainty in the estimate for each gradient component is as low as desired.
It is easy to see from the above procedure that the variance in the estimate of each individual gradient component does not have an explicit dependence on the number of elementary steps. This can be seen from Equation (30), which only has an explicit dependence on 3 points in the chain. Alternatively, from our assumptions designed to mirror the case of quantum circuits, we know the variance of
these estimators is controlled by the value of the product  $(i_j | H_j U_j | i_{j-1}) O(i_N) \leq 1$ , independent of the number of parameters or steps in the sampling process. It may appear that the quantity estimated could be unbounded, but if we move the denominator into  $p$ , the result is again a probability distribution multiplied only by values determined by the numerator here. As a result, analogous to backpropagation in the bitwise function case, by storing the intermediate configurations  $\{|i_j)\}$  at a cost memory of  $O(M)$ , we see that evaluating the gradient requires a number of samples that is independent of  $M$ .
From this, we see that indeed the desired scaling is possible in the case of the analogous classical parameterized stochastic processes on tensor product spaces. The formulation as a sum over paths also allows us to make connection to the gentle measurement results in the main text, in that we are always promised to be in a computational basis state, making it possible to do a gentle measurement at intermediate steps with unit probability. This division allows us to help identify the origin of challenges in achieving backpropagation scaling as a problem with quantum measurement collapse and the inability to read out intermediate states while continuing a computation, rather than the probabilistic formulation of the problem. In addition, one may make the classical generators  $H_{i}$  non-commutative with each other and suffer no additional difficulties in estimating the gradient components, unlike in the quantum case. It remains an interesting question to better understand the performance separation on practical tasks between quantum variational methods and this type of classical analog, given the advantage in trainability of the classical construction.
C Polynomial complexity circuits
It is reasonable to ask if we can first rule out backpropagation when only given access to single copies of a state. A useful tool to rule out the possibility of certain tasks is information-theoretic bounds, however, we show here that these are not sufficient to rule out quantum backpropagation scaling on single copies as the task remains information-theoretically viable under the assumption of a polynomial length variational circuit, thanks to classical shadows. On the other hand, standard computational arguments illustrate the difficulty in achieving the desired scaling.
C.1 Information-efficiency with classical shadows
The idea behind classical shadows is to create a classical representation of a state  $\rho$ , that allows one to affordably estimate other properties of interest, like expectation values of observables [Huang et al., 2020]. In general, the number of samples,  $N$ , needed to predict say,  $\mathrm{Tr}[E_1\rho],\ldots,\mathrm{Tr}[E_K\rho]$  within additive error  $\varepsilon$ , with high probability is
$$
N = \Omega (\log (K) \max _ {i} \| E _ {i} \| _ {\mathrm {s h a d o w}} / \epsilon^ {2}),
$$
where  $\| E_i\|_{\mathrm{shadow}}$  is a norm influenced by the particular measurement primitive chosen to implement the classical shadow scheme. While general quantum states can be hard to determine, the additional constraint of a state being generated by a polynomial complexity variational circuit allows us to strengthen our statements.
Definition 15 (Polynomial complexity circuit). We say a circuit is a polynomial complexity circuit if it is composed from a fixed gate set  $G$  that may be applied between any two qubits with a maximum number of gates scaling polynomially in  $n$ , the number of qubits. Additionally, we will call it a polynomial complexity parameterized circuit if each gate in the elementary set is defined by a bounded number of parameters.
With this at hand, we have the following.
Proposition 16 (Information-efficiency of polynomial complexity circuits). Let  $\rho = |\psi\rangle\langle\psi|$  be the density matrix of a pure state generated from a quantum circuit of polynomial complexity built from a gate set of size  $G$  applied between any two qubits, with at most  $p(n)$  total gates, where  $p(n)$  is a polynomial in the number of qubits,  $n$ . With these definitions, there are at most  $K = (nG)^{2p(n)}$  of these circuits. Then,  $\rho$  can be explicitly determined using  $\Omega(\log(K)/\varepsilon^2) = \Omega(2p(n)\log(nG)/\varepsilon^2)$  single-copy measurements and a classical search procedure.
Proof. Given that  $|\psi \rangle$  is generated from a polynomial complexity circuit, denote the possible states created by such a circuit as  $|\phi_i\rangle$ . With the above definitions it is easy to see that the total number of
possible states that can be generated by a single step is  $n^2 G$ , and hence with  $p(n)$  possible choices, the total number of states is  $K = (nG)^{2p(n)}$ . If the underlying set of operations used to generate the state is unknown, it is still possible to cover the space of two-qubit operations to diamond distance error  $\epsilon$  with a number of operations scaling polynomially in  $1 / \epsilon$  and  $p(n)$  [Caro et al., 2022]. If we denote this number of extended operations as  $G'$ , then the argument proceeds as before in terms of asymptotic scaling by replacing  $G$  with  $G'$ . Performing Clifford classical shadows with  $E_i = |\phi_i\rangle\langle\phi_i|$  for  $i = 1, \dots, K$ , one can estimate the fidelity, i.e.  $\mathrm{Tr}[E_i|\psi\rangle\langle\psi|]$ , for all  $i$  within additive error  $\varepsilon$  using  $\Omega(\log(K)/\varepsilon^2)$  single copies of  $|\psi\rangle$ . Since  $|\psi\rangle$  is generated by one of the  $K$  circuits, searching for an  $E_i$  that provides the maximum fidelity, allows one to find  $\mathrm{Tr}[E_i|\psi\rangle\langle\psi|] = 1$ , with high probability, and thus, explicitly determine  $|\psi\rangle$ , and a circuit that generated it by using classical simulation of the family of circuits, that will generally scale both exponentially in  $n$  and  $K$ .
With this knowledge, one may proceed to compute expectation values classically to determine gradients or indeed any desired expected value or feature of the state. Whilst this procedure allows us to determine  $|\psi \rangle$  and a circuit for creating it, executing it incurs quantum hardware costs dominated by the Clifford circuits needed for the classical shadow protocol – which are of polynomial depth, but contain entangling gates which are limiting in practice. Even more concerning, is the classical cost of post-processing. Obtaining the maximum fidelity involves storing  $K = (n + p(n))^{O(p(n))}$  values and searching over them, which can be expensive. Additionally, the final computation of the expectation values needed for backpropagation, requires knowing and storing  $M$  exponentially large matrices, over and above the cost to compute the expectation values. And so, backpropagation scaling remains untenable with this implementation.
C.2 Computational hardness on polynomial complexity circuits
The result and algorithm (a brute force search) used in Proposition 16 demonstrate the information-theoretic efficiency of determining almost anything one would want to know about a state if we are guaranteed that it is both a pure state and generated by a polynomial complexity circuit. The classical computational procedure is clearly inefficient, but this begs the question of whether an efficient procedure might exist in general, especially given the existence of an efficient procedure for special cases. Here we argue that no efficient procedure can exist in the most general case, unless it is possible to efficiently clone pseudo-random quantum states.
Proposition 17 (Computational hardness of polynomial complexity circuits). Under standard cryptographic assumptions, no efficient computational procedure exists to identify a pure state of polynomial complexity to trace distance  $\varepsilon$ .
Proof. A pseudo-random quantum state is defined to be a pure state of polynomial complexity that no efficient computational algorithm given a polynomial number of copies of the state can distinguish from the Haar random state. Using the procedure described in Proposition 16, a circuit that can recreate the state to trace distance  $\epsilon$  can be found using a polynomial number copies of the state. If the procedure that finds this circuit is also computationally efficient, then the state can be cloned efficiently, violating the no-cloning theorem for pseudo-random states shown in Ji et al. [2018], which merely rests upon standard cryptographic assumptions.
This result demonstrates that even if we know a state is a pure state generated from a polynomial complexity circuit, it is computationally infeasible to identify it under cryptographic assumptions despite the information-theoretic efficiency. This suggests that there are states and observables for which the backpropagation problem could remain challenging, and that the most effective strategies must make use of known structure in the observables and states to achieve computational efficiency in analogy to known special cases.
D Shadow tomography protocol for gradients
For much of this manuscript it has been assumed that one has complete white-box access to the input state  $\rho = |\psi (\theta)\rangle \langle \psi (\theta)|$ . In a more traditional quantum setting, however, this may not be the case. One may be given access to unknown quantum states, or partially unknown states, and tasked to process them for some machine learning task. In such an instance, the input states are usually referred to as
quantum data, and insights pertaining to this model set up can be found in Huang et al. [2021]. In this section, we discuss some details around this model type, which we call a quantum neural network and is defined in Definition (8).
D.1 Gradients as observables
Before presenting our algorithm for performing quantum backpropagation, we begin with the following remark on quantum neural networks which allows us to exploit a shadow tomography procedure.
Remark 18 (Gradient of a quantum neural network). The  $k^{\mathrm{th}}$  gradient component of the quantum neural network may be expressed as
$$
\begin{array}{l} \partial_ {\theta_ {k}} \operatorname {Q N N} _ {\bar {\theta}} (| \varphi \rangle) = 2 \operatorname {R e} \langle 0 | \langle \varphi | \mathcal {U} ^ {\dagger} (\vec {\theta}) Z _ {0} \partial_ {\theta_ {k}} \mathcal {U} (\vec {\theta}) | 0 \rangle | \varphi \rangle \\ = 2 \operatorname {R e} \left\langle \Phi_ {k} \mid \Psi_ {k} \right\rangle \\ \end{array}
$$
$$
\begin{array}{l} \left| \Psi_ {k} \right\rangle = \left(i P _ {k}\right) e ^ {i \theta_ {k} P _ {k}} U _ {k} \dots e ^ {i \theta_ {1} P _ {1}} U _ {1} | 0 \rangle | \varphi \rangle \\ = e ^ {i \left(\theta_ {k} + \frac {\pi}{2}\right) P _ {k}} U _ {k} \dots e ^ {i \theta_ {1} P _ {1}} U _ {1} | 0 \rangle | \varphi \rangle \\ \end{array}
$$
$$
\left| \Phi_ {k} \right\rangle = U _ {k + 1} ^ {\dagger} e ^ {- i \theta_ {k + 1} P _ {k + 1}} \dots U _ {M} ^ {\dagger} e ^ {- i \theta_ {M} P _ {M}} Z _ {0} e ^ {i \theta_ {M} P _ {M}} U _ {M} \dots e ^ {i \theta_ {1} P _ {1}} U _ {1} | 0 \rangle | \varphi \rangle .
$$
If one defines
$$
\begin{array}{l} \mathcal {U} _ {k} ^ {(\Psi)} = e ^ {i \left(\theta_ {k} + \frac {\pi}{2}\right) P _ {k}} U _ {k} \dots e ^ {i \theta_ {1} P _ {1}} U _ {1}, \\ \mathcal {U} _ {k} ^ {(\Phi)} = U _ {k + 1} ^ {\dagger} e ^ {- i \theta_ {k + 1} P _ {k + 1}} \ldots U _ {M} ^ {\dagger} e ^ {- i \theta_ {M} P _ {M}} Z _ {0} e ^ {i \theta_ {M} P _ {M}} U _ {M} \ldots e ^ {i \theta_ {1} P _ {1}} U _ {1}, \\ \end{array}
$$
then, given a copy of  $|\varphi \rangle$ , one may attach an ancilla qubit labelled * in the  $| + \rangle$  state (in addition to the output qubit 0). In doing so, consider applying control- $\mathcal{U}_k^{(\Psi)}$  conditional on the ancilla being  $|0\rangle$ , and control- $\mathcal{U}_k^{(\Phi)}$  conditional on the ancilla being  $|1\rangle$ . This produces the state
$$
\frac {1}{\sqrt {2}} \big (| 0 \rangle | \Psi_ {k} \rangle + | 1 \rangle | \Phi_ {k} \rangle \big).
$$
Measuring  $X$  on the ancilla qubit, the expectation is
$$
\begin{array}{l} \frac {1}{2} \left(\langle 0 | \langle \Psi_ {k} | + \langle 1 | \langle \Phi_ {k} |\right) X _ {*} \left(| 0 \rangle | \Psi_ {k} \rangle + | 1 \rangle | \Phi_ {k} \rangle\right) = \operatorname {R e} \langle \Phi_ {k} | \Psi_ {k} \rangle \\ = \frac {1}{2} \partial_ {\theta_ {k}} \mathrm {Q N N} _ {\vec {\theta}} (| \varphi \rangle). \\ \end{array}
$$
This implicitly gives an operator on  $| + \rangle | 0 \rangle |\varphi \rangle$  whose expectation value is  $\frac{1}{2}\partial_{\theta_k}\mathrm{QNN}_{\vec{\theta}}(|\varphi \rangle)$ . Moreover, we can implement this measurement with  $O(M)$  quantum operations.
In order to prove Theorem 9, we need to discuss and modify two concepts: online learning and threshold search [Aaronson et al., 2018, Bădescu and O'Donnell, 2021].
D.2.1 Online learning of quantum states
As in Aaronson et al. [2018], suppose we have access to a stream  $(E_1, b_1), \ldots, (E_M, b_M)$  where each  $b_k = \langle \psi | E_k | \psi \rangle$ . We want to compute hypothesis states  $\omega_1, \ldots, \omega_M$ , which are mixed states stored in classical memory, such that
-  $\omega_{k}$  depends only on  $(E_{1},b_{1}),\ldots ,(E_{k - 1},b_{k - 1})$  (the online condition)
-  $\left| {\operatorname{Tr}\left( {{E}_{k}{\omega }_{k}}\right)  - \langle \psi  \mid  {E}_{k} \mid  \psi }\right|  > \varepsilon$  for as few  $k$  as possible
One may produce the following theorem.
Theorem 19. [Aaronson et al., 2018, Theorem 1] In the above setting, there is an explicit strategy for outputting hypothesis states  $\omega_{1},\ldots ,\omega_{M}$  such that  $|\operatorname {Tr}(E_k\omega_k) - \langle \psi |E_k|\psi \rangle | > \varepsilon$  for at most  $O(\frac{n}{\varepsilon^2})$  values of  $k$ . This holds even if the measurements  $b_{k}$  are noisy, and only satisfy  $|b_{k} - \langle \psi |E_{k}|\psi \rangle |\leq \frac{\varepsilon}{3}$
Two remarks are in order: first, the problem setup and algorithm presented in Theorem 19 are both completely classical. Second, this theorem says nothing about computational runtime. Implementation of the algorithm in Theorem 19 using techniques from convex optimization will require runtime polynomial in the dimension of the Hilbert space  $\mathrm{poly}(2^n)$ .
D.2.2 Quantum Threshold Search
Bădescu and O'Donnell [2021] promote online learning to a shadow tomography protocol using a procedure which they call threshold search. This gives an improved version of the quantum private multiplicative weights algorithm proposed in Aaronson and Rothblum [2019]. The difference between the online learning setting from the previous section and general shadow tomography, is that in practice, we are typically not given the expectation values  $\{b_k\}$  and must measure them ourselves. This is where threshold search comes in handy. Suppose we possess some copies  $|\psi\rangle^{\otimes m}$  of a quantum state and are given a stream  $(E_1, a_1), \ldots, (E_M, a_M)$  where each  $a_k$  is supposed to be a guess such that  $a_k \approx \langle \psi | E_k | \psi \rangle$ . Threshold search is a subroutine which, given only logarithmically many copies of the state, can check in an online fashion whether there is an  $a_k$  which errs by more than  $\varepsilon$ . More formally, we have the following theorem.
Theorem 20. [Bădescu and O'Donnell, 2021, Lemma 5.2] Given  $m$  copies of an  $n$ -qubit quantum state  $|\psi\rangle^{\otimes m}$ ,  $M$  observables  $-1 \leq E_1, \ldots, E_M \leq 1$ , and guesses  $a_1, \ldots, a_M$ , there is an algorithm which outputs either
-  $|a_k - \langle \psi |E_k|\psi \rangle |\leq \varepsilon \forall k.$
-  $\text{Or} |a_k - \langle \psi |E_k|\psi \rangle| > \frac{3}{4}\varepsilon$  when in fact  $|b_{k} - \langle \psi |E_{k}|\psi \rangle |\leq \frac{1}{4}\varepsilon$  for a particular  $k$  and value  $b_{k}$ .
It does so using number of copies only
$$
m = O \left(\frac {\log^ {2} M}{\varepsilon^ {2}}\right).
$$
Furthermore, the algorithm is online in the sense that:
- The algorithm is initially given only  $M$  and  $\varepsilon$ . It then selects  $m$  and obtains  $|\psi \rangle^{\otimes m}$ .
- Next, observable/threshold pairs  $(E_1, a_1), (E_2, a_2), \ldots$  are presented to the algorithm in sequence. When each  $(E_k, a_k)$  is presented, the algorithm must either 'pass', or else halt and output  $|a_k - \langle \psi | E_k | \psi \rangle| > \frac{3}{4} \varepsilon$ .
- If the algorithm passes on all  $(E_k, a_k)$  pairs, then it ends by outputting  $|a_k - \langle \psi | E_k | \psi \rangle| \leq \varepsilon \forall k$
We stress that this subroutine requires quantum memory and multi-copy measurements, and uses gentle measurements in an essential way. One is able to check whether or not  $a_{k}$  is inside the threshold without greatly disturbing the copies of the quantum state. We are now ready to state the full shadow tomography protocol from Bădescu and O'Donnell [2021]. The idea is to run the online learning algorithm from Theorem 19 in parallel with threshold search, and Bădescu and O'Donnell [2021, Theorem 1.4] tells us that this algorithm succeeds in outputting estimates  $|b_{k} - \langle \psi |E_{k}|\psi \rangle |\leq \varepsilon$  with high probability.
When applying Algorithm 1 to the observables corresponding to gradients described in Appendix D.1, we can exploit that the observables are related sequentially. In between each round  $k$ , we rotate both, the states stored in quantum memory and the classical online learner, so that implementing the measurement of the next gradient only requires runtime independent of  $M$ . Since these rotations are unitary and do not reduce the quality of any approximations, the same proof as Bădescu and O'Donnell [2021, Theorem 1.4] will apply. This establishes Theorem 9.
By Bădescu and O'Donnell [2021, Theorem 1.4], this algorithm obtains estimates  $|b_{k} - \frac{1}{2}\partial_{\theta_{k}}\mathrm{QNN}_{\vec{\theta}}(|\varphi \rangle)| \leq \varepsilon$  for each  $k$  by taking the number of copies to be
$$
m = O \left(\frac {n \log^ {2} M}{\varepsilon^ {4}}\right).
$$
Algorithm 1 Online and gentle shadow tomography
Input:  $m$  copies of the unknown input state  $|\psi \rangle^{\otimes m}$ , in  $m$  registers each with  $n$  qubits.
Output: Estimates  $b_{k} \approx \langle \psi | E_{k} | \psi \rangle$
1. Set  $R = O\left(\frac{n}{\varepsilon^2}\right)$  and  $m_0 = O\left(\frac{\log^2 M}{\varepsilon^2}\right)$ . We need  $R$  batches, each with  $m_0$  copies, so  $m = Rm_0$  copies in total. This gives in total
$$
m = O \left(\frac {n \log^ {2} M}{\varepsilon^ {4}}\right)
$$
2. Initialize the online learner  $\omega_{1}$  according to the online learning algorithm.
3. Start with the first batch of copies  $|\psi \rangle^{\otimes m_0}$
4. For each  $k = 1,\dots ,M$  ..
(a) Use the online learner to predict  $a_{k} = \mathrm{Tr}(E_{k}\omega_{k})$
(b) Use threshold search to check  $|a_{k} - \langle \psi |E_{k}|\psi \rangle |$ .
(c) If threshold search passes  $|a_{k} - \langle \psi |E_{k}|\psi \rangle |\leq \varepsilon$
i. Output estimate  $b_{k} \gets a_{k}$ .
ii. Leave the online learner unchanged  $\omega_{k + 1}\gets \omega_k$
(d) If threshold search concludes  $|a_{k} - \langle \psi |E_{k}|\psi \rangle | > \frac{3}{4}\varepsilon$  and in fact  $|b_{k} - \langle \psi |E_{k}|\psi \rangle |\leq \frac{1}{4}\varepsilon$
i. Output estimate  $b_{k}$ .
ii. Update online learner with  $b_{k} \approx \langle \psi | E_{k} | \psi \rangle$  to get  $\omega_{k + 1}$ .
iii. Discard the current batch and move onto a fresh batch  $|\psi \rangle^{\otimes m_0}$
Moreover, the required number of quantum operations is
$$
O (m M) = O \left(\frac {n M \log^ {2} M}{\varepsilon^ {4}}\right)
$$
This is quasi-linear in  $M$ . With naive storage of the entire density matrix of the hypothesis state  $\omega_{k}$ , the classical cost is
$$
M \cdot 2 ^ {O (n)}
$$
Which is also linear in  $M$ , but unfortunately exponential in the input size  $n$ . We present the full algorithm for gradient estimation using online shadow tomography with threshold search in Algorithm 2.
Algorithm 2 Shadow tomography protocol for gradients of a quantum neural network
Input:  $m$  copies of the unknown input state  $|\varphi \rangle^{\otimes m}$  in  $m$  registers each with  $n$  qubits.
Output: Estimates  $b_{k} \approx \frac{1}{2}\partial_{\theta_{k}}\mathrm{QNN}_{\tilde{\theta}}(|\varphi \rangle)$  for  $k = 1,\ldots ,M$
1. Set  $R = O\left(\frac{n}{\varepsilon^2}\right)$  and  $m_0 = O\left(\frac{\log^2 M}{\varepsilon^2}\right)$ . We need  $R$  batches, each with  $m_0$  copies, so  $m = Rm_0$  copies in total. This gives
2. Define for each  $k = 1,\dots ,M$
$$
\left| \psi_ {k} \right\rangle = \frac {1}{\sqrt {2}} \left(\left| 0 \right\rangle \left| \Psi_ {k} \right\rangle + \left| 1 \right\rangle \left| \Phi_ {k} \right\rangle\right)
$$
and recall from Remark 18 that
$$
\langle \psi_ {k} | X _ {*} | \psi_ {k} \rangle = \frac {1}{2} \partial_ {\theta_ {k}} \mathrm {Q N N} _ {\vec {\theta}} (| \varphi \rangle)
$$
3. Attach the output qubit and an ancilla qubit in the  $| + \rangle$  state to each register. Label the output qubit 0 and the ancilla qubit *.
4. To each register, do the following: