SlowGuess/NeurIPS_2023_OCR huggingface dataset:
(a) Apply control- $\mathcal{U}_1^{(\Psi)}$  conditional on the ancilla being  $|0\rangle$ . This requires  $O(1)$  quantum operations.
(b) Apply control- $\mathcal{U}_1^{(\Phi)}$  conditional on the ancilla being  $|1\rangle$ . This requires  $O(M)$  quantum operations. This step is analogous to the initial forward pass in classical backpropagation.
(c) This produces the state  $|\psi_1\rangle \otimes^m$
5. Initialize the online learner  $\omega_{1}$  according to the online learning algorithm.
6. Start with the first batch of copies  $|\psi_1\rangle^{\otimes m_0}$
7. For  $k = 1, \ldots, M$ , do the following. This loop is analogous to the backward pass in classical backpropagation.
(a) Use the online learner to predict  $a_{k} = \operatorname{Tr}(X_{*}\omega_{k})$ .
(b) Use threshold search to check  $|a_{k} - \langle \psi_{k}|X_{*}|\psi_{k}\rangle|$ . This takes time independent of  $M$ .
(c) If threshold search passes  $|a_{k} - \langle \psi_{k}|X_{*}|\psi_{k}\rangle |\leq \varepsilon$
(d) If threshold search concludes  $|a_{k} - \langle \psi_{k}|X_{*}|\psi_{k}\rangle| > \frac{3}{4}\varepsilon$  and in fact  $|b_{k} - \langle \psi_{k}|X_{*}|\psi_{k}\rangle| \leq \frac{1}{4}\varepsilon$ ,
i. Output estimate  $b_{k}$ .
ii. Update online learner with  $b_{k} \approx \langle \psi_{k}|X_{*}|\psi_{k}\rangle$  to get  $\omega_{k + 1}$ .
iii. Discard the current batch and move onto a fresh batch.
(e) To each register in the current batch and the unused batches, do the following:
i. Apply control-  $(e^{i(\theta_{k + 1} + \frac{\pi}{2})P_{k + 1}}U_{k + 1}e^{-i\frac{\pi}{2} P_k})$  conditional on the ancilla being  $|0\rangle$  This implements  $\mathcal{U}_{k + 1}^{(\Psi)}(\mathcal{U}_k^{(\Psi)})^{-1}$  , and only requires  $O(1)$  quantum operations.
ii. Apply control- $e^{i\theta_{k+1}P_{k+1}}U_{k+1}$  conditional on the ancilla being  $|1\rangle$ . This implements  $\mathcal{U}_{k+1}^{(\Phi)}(\mathcal{U}_k^{(\Phi)})^{-1}$ , and only requires  $O(1)$  quantum operations.
iii. This produces in each batch (a noisy approximation to) the state  $|\psi_{k + 1}\rangle^{\otimes m_0}$ .
(f) Also apply the rotations in Step (e) to the hypothesis state  $\omega_{k + 1}$  in classical memory.
The online learner now approximates  $|\psi_{k + 1}\rangle \langle \psi_{k + 1}|$
E Fully gentle gradient estimation
In this section, we motivate for a need to perform sequential and gentle measurements to individual gradient states, as opposed to superpositions of them. Thereafter, we discuss general strategies based on gentle measurements alone, performed on single and multiple copies.
E.1 Considering individual gradient states
While we briefly motivated the need for a sequential reuse of information in measurements in the main text, here we further motivate such a construction as a necessary, but perhaps not sufficient condition for our purposes. Given that one can create a superposition over all the potential gradient components at a cost that only requires a single function call, it is natural to ask if this ability gives us any headway in achieving our goals. Consider exploiting the superposition over all gradient states
$$
| \Psi \rangle = \sum_ {k = 1} ^ {M} c _ {k} | A _ {k} \rangle \prod_ {j \in A} U _ {j} | 0 \rangle = \sum_ {k = 1} ^ {M} c _ {k} | A _ {k} \rangle | \psi_ {k} \rangle , \tag {31}
$$
using at most  $cM$  calls to the family  $\{U_j\}$  and some ancillary qubits  $|A_k\rangle$  associated with the  $k^{\mathrm{th}}$  gradient state.2 Creating such a superposition weakens our ability to extract each gradient component's signal upon measurement, and thus, requires more samples to distinguish between gradient components with a desired precision. From a cost perspective, it remains optimal or equivalent to consider gradient states  $|\psi_k\rangle$  individually. To make this more concrete, consider a state discrimination task, with the following lemma at hand.
Lemma 21 (Optimal two-state discrimination). Any quantum algorithm that distinguishes two states  $\rho_{1}$  and  $\rho_{2}$  using a single copy of each state with probability at least 0.9 requires
$$
\frac {1}{2} + \frac {1}{2} \| \rho_ {1} - \rho_ {2} \| _ {\mathrm {t r}} \geq 0. 9. \tag {32}
$$
Now we may proceed to the state discrimination task, where it is clear a superposition is not helpful.
Proposition 22. Consider the two-state discrimination task for two scenarios. First, given  $|\psi_m\rangle$  and  $|\phi_m\rangle$ , where  $\langle \psi_m|\phi_m\rangle = 0$ , there is a measurement strategy that can distinguish the states with a single measurement. Second, given the states
$$
| \Psi \rangle = \frac {1}{\sqrt {M}} \sum_ {k = 1} ^ {M} | A _ {k} \rangle | \psi_ {k} \rangle , \tag {33}
$$
$$
\left| \Phi \right\rangle = \frac {1}{\sqrt {M}} \sum_ {k = 1} ^ {M} \left| A _ {k} \right\rangle \left| \phi_ {k} \right\rangle , \tag {34}
$$
where  $|\psi_k\rangle = |\phi_k\rangle$  for every  $k$  except the  $m^{th}$  component and  $\langle \psi_m|\phi_m\rangle = 0$  as before, then  $\Omega (M)$  copies are required by any strategy aiming to discriminate  $|\Psi \rangle$  from  $|\Phi \rangle$  with reasonably high success probability.
Proof. The first scenario follows straightforwardly from Lemma (21) since there is no overlap between  $|\psi_m\rangle$  and  $|\phi_m\rangle$  - hence, their trace distance is 1 and Equation (32) always holds. For states in uniform superposition over all  $M$  components, the overlap is  $1 - 1 / M$  which is close to unity for large  $M$ , indicating the difficulty of the task when the states mostly overlap. Given access to  $N$  copies of  $|\Psi \rangle$  and  $|\Phi \rangle$ , to discriminate with probability at least 0.9 requires
$$
\frac {1}{2} + \frac {1}{2} \sqrt {1 - | \langle \Psi | \Phi \rangle | ^ {2 N}} \geq 0. 9, \tag {35}
$$
$$
\left(1 - \frac {1}{M}\right) ^ {2 N} \leq 0. 3 6, \tag {36}
$$
implying that  $N = \Omega (M)$  in order to discriminate successfully with the desired probability.
${}^{2}c$  is some small constant.
From Proposition (22), we have the immediate corollary.
Corollary 23. It is either optimal or equivalent in cost to consider gradient states individually, as opposed to a superposition over them all.
Proof. Replacing the uniform superposition in Equations (33) and (34) to the more general,  $|\Psi \rangle = \sum_{k=1}^{M} c_k |A_k\rangle |\psi_k\rangle$  and  $|\Phi \rangle = \sum_{k=1}^{M} c_k |A_k\rangle |\phi_k\rangle$ , the number of samples needed to discriminate the  $m^{\text{th}}$  component scales as  $N \sim 1/c_m^2$ . Since  $c_m^2 \in [0,1]$ , it is clear that  $c_m^2 = 1$  is optimal. If there are  $M$  components, then  $c_m^2 \sim 1/M$  and hence,  $N \sim M$ . Assuming the superposition state  $|\Psi \rangle$  incurs a cost proportional to  $M$ , the number of samples required to differentiate between components in the wave function will imply an overall cost that scales as  $M^2$ .
E.2 A case for sequential and gentle measurement
Whilst the cost equivalence presented in Corollary 23 implies no benefit from a superposition of gradient states, it also suggests that, if one is to obtain backpropagation scaling, individual gradient states must be utilized in a more resource efficient manner. Drawing inspiration from backpropagation, if one could instead use the state  $|\psi_k\rangle$  to make a measurement, then update it to  $|\psi_{k + 1}\rangle$  without substantially disturbing it, it would then be possible to perform all of the measurements at an overall cost scaling like  $O(M)$ . We illustrate such a benefit by means of an example using fictitious non-destructive measurements in Algorithm 3.
Algorithm 3 Gradient estimation with a modified, non-destructive swap test
Input: Three registers initialized to  $| + \rangle | 0\rangle | 0\rangle$
Output: Gradient vector estimate for  $F(\theta)$
1. Apply  $U(\theta) = U_M \dots U_1$  to the second register, controlled on the first being 0. Cost  $\sim M$ .
2. Apply  $OU(\theta)$  to the third register, conditional on the first being 1. Cost  $\sim M$  and the state becomes
$$
\left. \left| + \right\rangle \left| 0 \right\rangle \left| 0 \right\rangle \rightarrow \frac {1}{\sqrt {2}} \left(\left| 0 \right\rangle \left| \psi_ {M} \right\rangle \left| 0 \right\rangle + \left| 1 \right\rangle \left| 0 \right\rangle \left| \lambda \right\rangle\right),
$$
where  $|\psi_M\rangle = U_M\ldots U_1|0\rangle$  and  $|\lambda \rangle = OU_{M}\ldots U_{1}|0\rangle$ . By assumption, all  $U_{j}$  and  $O$  are hermitian and unitary.
3. For  $k$  in  $\{M,M - 1,\dots,1\}$
(a) Apply and update  $|\psi_k\rangle = -iP_k|\psi_k\rangle$  conditioned on ancilla being 0. Cost  $\sim 1$ .
(b) Perform a non-destructive swap test on the state
$$
\frac {1}{\sqrt {2}} \left(| 0 \rangle | \psi_ {k} \rangle | 0 \rangle + | 1 \rangle | 0 \rangle | \lambda \rangle\right)
$$
to estimate  $[F'(\theta)]_{\theta_k} = -2 \operatorname{Im}\langle \lambda |\psi_k\rangle$  with no damage to the state. Cost  $\sim 1$ .
(c) If  $k > 1$  apply and update  $|\lambda \rangle = U_k^\dagger |\lambda \rangle$  conditional on ancilla being 1. Cost  $\sim 1$ .
(d) If  $k > 1$  apply and update  $|\psi_{k - 1}\rangle = U_k^\dagger (iP_k)\left|\psi_k\right\rangle$  conditional on ancilla being 0. Cost  $\sim 1$ .
The procedure naturally breaks down in a real quantum computer at Step (3b) due to the reliance on non-destructive measurements. Substituting these for gentle measurements, which are only partially non-destructive but, at least, theoretically possible, one may still aspire to exploit the structure of the problem and achieve backpropagation scaling as in Algorithm 3.
E.3 Gentle measurement on single copies
The need to reuse a state enough times to extract every gradient component, imposes constraints on the gentleness of measurements made. While the use of multiple copies may enhance the ability to leverage gentle measurements, it is straightforward to see why this approach would not work in general, when given access to a single copy of  $\rho$ . Using a scheme like the modified swap test in
Algorithm 3, implies that each measurement must be on average  $1 / M$ -gentle in order to reuse the state  $M$  times to extract each gradient component without damaging the state to the point that at least one observable on the state is completely wrong. Enforcing such a constraint, leads to measurements that are trivial - i.e. they barely depend on  $\rho$  and cannot yield enough information about gradients. We recap some useful lemmas whose proofs can be found in Aaronson and Rothblum [2019] to make this more concrete.
Lemma 24 (Additivity of damage). Let  $\rho$  be some mixed state and let  $S_{1}, S_{2}, \ldots, S_{M}$  be general quantum operations. Suppose for all  $j$ , we have
$$
\| S _ {j} (\rho) - \rho \| _ {\mathrm {t r}} \leq \alpha_ {j},
$$
$$
\left\| S _ {M} \left(S _ {M - 1} (\dots S _ {1} (\rho))\right) - \rho \right\| _ {\mathrm {t r}} \leq \alpha_ {1} + \dots + \alpha_ {M}.
$$
Lemma 25 (Trivial measurement). Given a measurement  $M$  and parameter  $\eta \geq 0$ , suppose that for every two orthogonal pure states  $|\psi\rangle$  and  $|\phi\rangle$ , and every possible outcome  $y$  of  $M$ , we have
$$
\Pr [ M (| \psi \rangle) \text {o u t p u t s} y ] \leq e ^ {\eta} \Pr [ M (| \phi \rangle) \text {o u t p u t s} y ].
$$
Then  $M$  is  $\eta$ -trivial. Further, let  $E_1 + \ldots + E_k = I$  be the POVM elements of  $M$ . Assume without loss of generality that the outcome  $y$  corresponds to the element  $E = E_1$ . Then,
$$
\langle \psi | E | \psi \rangle \leq e ^ {\eta} \langle \phi | E | \phi \rangle ,
$$
holds for all states, not just all orthogonal  $|\psi \rangle ,|\phi \rangle$
Lemma 26 (Triviality lemma). Suppose a measurement is  $\alpha$ -gentle on all states. Then the measurement is  $\ln \left(\frac{1 + 4\alpha}{1 - 4\alpha}\right)$ -trivial — so in particular,  $O(\alpha)$ -trivial, provided  $\alpha \leq \frac{1}{4.01}$ .
Equipped with these lemmas, we proceed to demonstrate the difficulty of gentle gradient estimation with single-copy access to a pure state.
Theorem 27. A sequence of  $M$  measurements on a single-copy pure state that is  $1 / M$ -gentle at every step to extract every gradient component, will be trivial.
Proof. Choose a circuit such that gradient state differs substantially, i.e.  $||\psi_i\rangle \langle \psi_i| - |\psi_j\rangle \langle \psi_j||_{\mathrm{tr}} = 1$  for all measurements. In other words, there is a unitary that must be applied to advance from gradient component  $i$  to  $j$ , otherwise there will be a measurement that produces the incorrect result if no such unitary is applied. Fix  $\{\Lambda ,\mathbb{I} - \Lambda \}$  as the POVM elements of a gentle measurement. Assume without loss of generality that the outcome of measuring the gradient component with respect to a given state corresponds to the element  $\Lambda = A^{\dagger}A$ , and
$$
\left\| S (\rho) - \rho \right\| _ {\mathrm {t r}} \leq \alpha \tag {37}
$$
$$
S (\rho) = \frac {A \rho A ^ {\dagger}}{\mathrm {T r} [ \Lambda \rho ]}.
$$
Using a single copy of  $\rho = |\psi\rangle\langle\psi|$  to extract all  $M$  gradient components, requires advancing the state after measuring gently at each step, and thus, each measurement step must be on average  $1/M$ -gentle to ensure
$$
\left\| S \left(U _ {M} S \left(U _ {M - 1} \dots S \left(U _ {2} S \left(U _ {1} \rho U _ {1} ^ {\dagger}\right) U _ {2} ^ {\dagger}\right) \dots U _ {M - 1} ^ {\dagger}\right) U _ {M} ^ {\dagger}\right) - \rho_ {M} \right\| _ {\mathrm {t r}} <   1, \tag {38}
$$
where  $\rho_{M}$  is the density matrix representation of the advanced gradient state  $|\psi_M\rangle = U_M\ldots U_2U_1|\psi \rangle$  If we allowed for any more damage at a particular step, we could eventually reach a point where subsequent measurements yield incorrect results, as the cumulative damage to the state may exceed 1. While the gentleness could be distributed across each gradient component in different ways, from the above lemma, we see that the more gentle the operator, the more trivial it becomes. Hence, if we had  $(M - 1)$  0-gentle measurements, they would be infinitely trivial and provide no information with 1 informative measurement. Hence, the least trivial set of measurements that achieve an average of  $1 / M$  gentleness would be to have each measurement be  $1 / M$  gentle. By Lemma (26), then each measurement will be  $O(1 / M)$ -trivial, which implies
$$
\operatorname {T r} \left[ \Lambda \rho_ {i} \right] \leq e ^ {1 / M} \operatorname {T r} \left[ \Lambda \rho_ {i + 1} \right]
$$
for any two gradient states  $\rho_{i},\rho_{i + 1}$ . As  $M$  increases, the estimates for all gradient components will converge. Therefore, the measurement operator has an exponentially vanishing dependence on the input states themselves and hence, provides little-to-no information about the gradient components.
E.4 Multiple copies and non-collapsing measurements
Non-adaptive, non-collapsing measurements are, by assumption, measurements that do not disturb the state of a quantum system at all. Under this assumption, the complexity class, non-adaptive Collapse-free Quantum Polynomial time (naCQP) was introduced. With this ability, searching through an unstructured  $M$ -element list can be performed in  $\tilde{O}(M^{\frac{1}{3}})$  time, which is faster than the optimal lower bound of  $O(M^{\frac{1}{2}})$  given by Grover's search algorithm [Grover, 1996]. Importantly, time complexity in naCQP is measured as the number of oracle queries plus the number of non-collapsing measurements. This definition is considered more fitting, since any task in naCQP allows for exponentially many non-collapsing measurements to be made and should thus, be accounted for.
Interestingly, one may still violate Grover's bound by allowing for approximately non-collapsing measurements. First, note that
$$
\| \rho - \rho^ {\prime} \| _ {\mathrm {t r}} = 0
$$
for non-collapsing measurements, where  $\rho'$  is the normalized state after measurement. In the approximately non-collapsing regime, assume that a measurement operator can be applied to a tensor product of the state  $\rho$  such that
$$
\left\| \rho^ {\otimes m} - \rho^ {\prime \otimes m} \right\| _ {\mathrm {t r}} \leq \alpha .
$$
As  $\alpha \to 0$ , we recover the non-collapsing measurement regime. In the gradient setting, approximately non-collapsing measurements are merely gentle measurements. This leads to the following.
Proposition 28. A sufficiently gentle measurement used for gradient extraction can solve an unstructured search problem in  $\tilde{O}(M^{\frac{1}{3}})$  time.
Proof. Reformulating the gentle gradient task as a search problem, let  $M = 2^n$ . Consider the state
$$
\sin ((2 i + 1) \theta) | x \rangle | 1 \rangle + \cos ((2 i + 1) \theta) \sum_ {y \in \{0, 1 \} ^ {n}, y \neq x} 2 ^ {- \frac {M - 1}{2}} | y \rangle | 0 \rangle \tag {39}
$$
after applying  $i = M^{\frac{1}{3}}$  Grover iterations, where  $|x\rangle$  is the marked state. The probability of measuring the marked state is  $|\sin ((2i + 1)\theta)|^2\approx 1 / M^{\frac{1}{3}}$ . Suppose we can create the state  $|\psi \rangle^{\otimes m}$ , where  $m = O(\log (M))$  by using  $M^{\frac{1}{3}}\log (M)$  Grover queries. By having access to multiple copies of  $|\psi \rangle$ , assume that one may implement a  $1 / M$ -gentle measurement on the copies as required for gradient estimation. Then, the probability of observing the marked state after a single gentle measurement is  $\log (M) / M^{\frac{1}{3}}$ . By performing  $M^{\frac{1}{3}}$  gentle measurements on the  $\log (M)$  copies, the probability of obtaining the marked state at least once is greater than  $1 - e^{-\log (M)} = 1 - \frac{1}{M}$ , using only  $\tilde{O}(M^{\frac{1}{3}})$  Grover oracle queries and  $O(M^{\frac{1}{3}})$  partially non-collapsing measurements, and thus, runs in time  $\tilde{O}(M^{\frac{1}{3}})$ .