SlowGuess/NeurIPS_2023_OCR huggingface dataset:
What comprises classical memory and time complexity, is purposely left vague. The details depend on the constituent types of operations needed to compute a function and its gradients, as well as the memory access model available. But, details aside, backpropagation merely refers to gradient computation in a particular manner, and, any reasonably successful implementation of it incurs a constant overhead in relative complexity, as captured by Equations (1) and (2). With this in mind, we elaborate on the operational definition of quantum backpropagation scaling in terms of memory. Thereafter, we explain the failure of various current gradient methods to achieve backpropagation scaling.
A.1 Memory complexity of the function
Recall the function of interest  $F(\theta) = F(\theta) = \mathrm{tr}[\rho (\theta)O]$ , where  $O$  is an observable and  $\rho (\theta)$  is a parameterized quantum state built from  $M$  parameters, acting either on an unknown initial state  $\rho$  or simplified initial state  $\rho = |0\rangle \langle 0|$ . Classifying the memory used to compute the function as a combination of  $n$  qubits, plus storage for each of the  $M$  parameters with appropriate precision,  $\delta$ , implies
$$
\operatorname {M E M O R Y} (F (\theta)) = \tilde {O} (n + M \log (1 / \delta)). \tag {10}
$$
To derive the computational cost, assume unit cost access to any element of the circuit family  $\{U_j\}$ . If an incoherent measurement scheme is used, measuring  $O$  and estimating  $F(\theta)$  to an acceptable fixed precision,  $\varepsilon$ , on repeated preparations of  $\rho(\theta)$  incurs a cost that scales as  $\mathrm{TIME}(F(\theta)) = \tilde{O} \left( \frac{M}{\varepsilon^k} \right)$ , for some integer  $k$ . This sets the scene for the computational requirements of computing  $F'(\theta)$ , which should, importantly, be achieved with a modest space overhead to truly replicate backpropagation.
A.2 Current gradient methods
Replicating classical backpropagation efficiency in a quantum setting requires more effort, which we elaborate on next by discussing how and why current gradient methods fail to achieve this efficiency. For further illustration, Figure 2 provides a hypothetical comparison between the popular gradient method – the parameter-shift rule – and true quantum backpropagation. The plot incorporates assumptions about time to compute native quantum operations taken from Babbush et al. [2021].
A.2.1 Naive sampling
The gradient of the function  $F(\theta)$  expressed in Equation (4) also takes a simpler form using the parameter-shift rule and properties of Pauli generators [Mitarai et al., 2018, Schuld et al., 2019]
$$
\left[ F ^ {\prime} (\theta) \right] _ {\theta_ {k}} = F \left(\theta + \frac {\pi}{2} \hat {\theta} _ {k}\right), \tag {11}
$$
where  $\hat{\theta}_k$  is a unit vector along the  $k^{\mathrm{th}}$  direction of  $\theta$ . Thus far, sampling schemes constructed to estimate (11), perform a destructive measurement that typically only retrieves a partial amount of information for one component of the gradient. As a result, reducing the infinity norm error in the gradient such that we expect  $||F'(\theta) - \hat{F}'(\theta)||_{\infty} \leq \varepsilon$  with reasonable probability, has a cost that scales like converging each component, i.e.
$$
\begin{array}{l} \operatorname {T I M E} \left(F ^ {\prime} (\theta)\right) \propto M \log M \operatorname {T I M E} (F (\theta)) (12) \\ = \tilde {O} \left(M ^ {2} / \varepsilon^ {2}\right). (13) \\ \end{array}
$$
While this quadratic dependence on the number of parameters may not seem problematic, a linear dependence was the necessary catalyst in the age of modern deep learning, with overparameterized networks that perform exceedingly well on practical tasks.
A.2.2 Fast gradient algorithm
A method put forth by Jordan [2005] numerically estimates the gradient of a classical black-box function at a given point, using a quantum computer. The algorithm impressively requires a single black-box query to estimate the full gradient with a desired precision, whilst satisfying the memory
Figure 2: Quantum backpropagation scaling. The parameter-shift rule is plotted alongside true quantum backpropagation scaling. On the  $x$ -axis is time in number of seconds required to compute a single estimate of the gradient in log scale, with common time points stated explicitly. On the  $y$ -axis is the number of parameters, also in log scale, that may be optimized using each method, for a given amount of time. We make simple assumptions, motivated from the work in Babbush et al. [2021]. Namely, we assume a minimum system size of  $n = 100$  qubits. Further, assuming a favourable time of  $10\mu s$  to compute one parameterised operation (which is 1 order of magnitude less than the time to compute one Toffoli gate), the time for one primitive is lower bounded by  $100 \times 10\mu s = T_{q}$ . Scaling in time is then roughly  $M^2 \cdot T_q$  for the parameter-shift rule and  $M \cdot \mathrm{polylog}(M) \cdot T_q$  for quantum backpropagation. Furthermore,  $\varepsilon = O(1)$ .
requirement in (2). We elaborate on the connection between this approach and backpropagation on a quantum computer when the function considered is classical and reversible, in Appendix B.1. But, as shown by Gilyén et al. [2019], when parameters are considered to be rotation angles like those in variational circuits, a different query model needs to be applied and the original single-query advantage becomes unattainable. With the appropriate query model, the known bounds imply a computational cost of  $\tilde{O}(M\sqrt{M} / \varepsilon^2)$  using amplitude estimation, and, in a high precision regime,  $\tilde{O}(M\sqrt{M} / \varepsilon)$  is worst-case optimal even with commuting Pauli operators [Huggins et al., 2021]. This worst-case bound was proved in a setting where operators commute, indicating that commutativity need not be helpful in other settings.
A.2.3 Simultaneous perturbation stochastic approximation (SPSA) algorithm
A few studies have investigated the use of the simultaneous perturbation stochastic approximation (SPSA) algorithm to optimize parameterized quantum circuits [Benedetti et al., 2019, Hoffmann and Brown, 2022, Gacon et al., 2021]. It is argued that SPSA is computationally efficient since its requires two function evaluations to estimate the gradient, irrespective of  $M$ . This seemingly satisfies the scaling we require, however, the approximation of the gradient has limited accuracy which affects the number of optimization steps needed for SPSA to converge to a minimum. As  $M$  increases, the variance of the gradient estimate increases and, thus, to counteract this, a smaller learning rate must be used - increasing the number of optimization steps - or more samples are needed to estimate the gradient with an appropriate accuracy at every step. In either case, one cannot escape a dependence on  $M$ , which indirectly affects the number of function evaluations needed to estimate gradients or perform gradient-based optimization adequately. More formally, the gradient estimator for component  $j$  of a function, given by SPSA, is
$$
\bar {F} ^ {\prime} (\theta) _ {j} = \frac {F (\theta + c \Delta) - F (\theta - c \Delta)}{2 c \Delta_ {j}} \tag {14}
$$
where  $c$  is a step size constant and  $\Delta \in \mathbb{R}^M$  is a size  $M$  random variable with independent, zero-mean, bounded second moments, and bounded inverse moments, i.e.  $\mathbb{E}(|\Delta|_j^{-1})$  is uniformly bounded for all
$j$ . A common choice for  $\Delta$  is a Bernoulli random variable with equal probabilities of being  $+1$  or  $-1$  for every entry.
Consider a special case,  $F$ , for pedagogical purposes such that the gradient at the point  $\theta$  is a constant  $g$  along all coordinates, the function is nearly linear at the point examined, and the number of coordinates  $M$  is large in a central limit theorem sense. We then have,  $F'(\theta)_j = g$  for all  $j$ , and  $F(\theta + c\Delta) \approx F(\theta) + cF'(\theta)^T\Delta = F(\theta) + cg\vec{1}^T\Delta \approx F(\theta) + \mathcal{N}(0, cgM)$ . On a quantum computer, the estimator will be constructed by taking independent measurements of  $F(\theta \pm c\Delta)$  and then rescaling the sample mean by  $1/2c\Delta_j$ . We then see that the variance of an individual term in this case is given by
$$
\operatorname {V a r} \left[ \bar {F} ^ {\prime} (\theta) _ {j} \right] = \frac {F (\theta)}{c} + g M \tag {15}
$$
As such the number of samples required to reach a precision  $\epsilon$  with high probability in even a single gradient component scales as
$$
N _ {s} = \frac {F (\theta) / c + g M}{\epsilon^ {2}} \tag {16}
$$
which clearly increases linearly with the number of components  $M$ , and does not achieve the desired scaling despite the estimator being constructed from only two function calls. It is also worth noting that the estimates for each component of the gradient are highly correlated across the vector, which can lead to larger errors than would be otherwise expected under alternative norms. This is intuitively expected, as it should not generally be possible to determine  $M$  independent random variables from a single value without increasing the precision of the estimates at least proportionately. We note in passing that generally to obtain an unbiased estimator one must also take  $c$  to be on the order of  $\epsilon$ , but this dependence can be improved with higher order formulas to  $\epsilon^{-k}$  for some  $k > 1$  [Spall, 2000], but this is not central to our study.
B Classical backpropagation in quantum circuits
In order to frame the discussion, it is worth considering a number of closely related setups as they would appear if performed on a quantum computer. In particular, in similar notation and cost models, its interesting to consider how classical backpropagation would look in a quantum circuit for a deterministic classical function and perhaps the closer classical analog, classical parameterized Markov processes on the space of probabilistic bits.
B.1 Classical functions
First we will look at an entirely classical function using reversible arithmetic for the purposes of analogy, using a simplified function but with simple generalizations available. This will be helpful for setting the stage in terms of notation and scaling, and also help make a connection with the gradient algorithm of Jordan [2005]. Consider a classical function  $f$  that depends on some set of parameters  $x \in \mathbb{R}^M$  via more elementary functions  $f_i$ . For this example, we assume a simple dependency graph for the overall function  $f: \mathbb{R}^M \to \mathbb{R}$  is the simple composition of elementary functions,  $f = f_N \circ f_{N-1} \circ \ldots \circ f_1$ . Given this structure, we denote a set of intermediate variables  $z_i$ , such that  $z_i = x_i$  for  $i \in [1, M]$  and  $z_i = f_i(z_{\alpha(i)})$  for  $i \in [M+1, n]$  where  $\alpha(i)$  is the subset of variables needed to evaluate  $f_i$ , noting that we are implicitly including a trivial set of elementary functions  $f_i$  that are simply the identity operation. We also assume that no  $z_i$  depends on itself, each  $z_i$  appears exactly once, and derivatives of the elementary operations are readily available, that is a simple function for evaluating  $f_i'(z)$  is available for any input  $z$ .
Given these definitions, we are ready to describe the algorithm for obtaining the gradient  $\nabla_{x}f(x)$ . We consider a universal precision  $\delta$  for all parameters and function values, such that classical numbers use  $O(\log (1 / \delta))$  qubits for their representation. For initialization, we store each of the parameters  $x_{i}$  in their own quantum register  $|\rangle_{x}$  to run the circuit fully within the quantum computer. In the first step, we run the function evaluation in the so-called forward pass and store the intermediate values  $z_{i}$  each in their own quantum register  $|\rangle_{z}$  using the elementary implementations of  $f_{i}$  as reversible circuits. Taking now an additional set of auxiliary registers,  $|\rangle_{\lambda}$  with the same size as the intermediate variables, we assign  $\lambda_{n} = 1$ , and compute the backwards pass according to reversible
implementations of  $\lambda_{j} = \sum_{i\in \beta (j)}\partial z_{j}f_{i}(z_{\alpha (i)})$  where  $\beta (i)$  is the outgoing nodes for intermediate variables  $z_{i}$ . In the final step, we may simply read off the  $\lambda$  register to find  $\nabla_x f(x) = \lambda_{1:M}$ .
Considering a general auxiliary register  $|\rangle_A$ , these steps may be written in quantum form as
$$
\left| \right. x \left. \right\rangle_ {x} \left| \right. 0 \left. \right\rangle_ {z} \left| \right. 0 \left. \right\rangle_ {\lambda} \left| \right. 0 \left. \right\rangle_ {A} \rightarrow^ {\text {F o r w a r d}} \left| \right. x \left. \right\rangle_ {x} \left| \right. z \left. \right\rangle_ {z} \left| \right. 0 \left. \right\rangle_ {\lambda} \left| \right. r _ {f} \left. \right\rangle_ {A} \rightarrow^ {\text {B a c k w a r d}} \left| \right. x \left. \right\rangle_ {x} \left| \right. z \left. \right\rangle_ {z} \left| \right. \lambda \left. \right\rangle_ {\lambda} \left| \right. r _ {b} \left. \right\rangle_ {A} \tag {17}
$$
where  $r_f$  and  $r_b$  denote the state of the arithmetic trash register after the forward and backwards pass respectively. Given our precision specification, the size of each of the  $x$  register is  $\tilde{O}(M)$  and the size of the  $z$  and  $\lambda$  registers are  $\tilde{O}(N)$ . This representation is a bit wasteful in that as the backwards pass proceeds one can overwrite the intermediate values  $z$  with  $\lambda$  when they are no longer needed, but writing it this way clarifies the steps. If we assume a typical setup where the number of free parameters is roughly on par with the number of elementary functions, then we see that the total storage for the primary registers is  $\tilde{O}(M)$  and similar for the ancillary register. Similarly, the amount of computation required in both the forward and backwards pass is  $\tilde{O}(M)$ , or approximately twice the cost of evaluating the function in the forward direction, meeting the scaling requirements of backpropagation with some small overhead for maintaining reversibility.
It is useful to compare some aspects of this approach to the quantum algorithm of Jordan for evaluating gradients of classical functions using a single black box function query [Jordan, 2005]. Considering only the computation, if we approximate the forward pass and backwards pass to each be the same cost as one black box function query, then up to log factors in precision of evaluation this method is a constant factor of two more expensive. Said another way, there is no quantum advantage in evaluating the gradient when one has white box access to the classical function implementation and it satisfies the simple dependencies requirements. In terms of storage requirements, the algorithm of Jordan requires the same  $x$  register, but makes no use of the intermediate variable registers such as  $z$  or  $\lambda$  (which can be combined in real implementations to be approximately the size of the  $x$  register). This use of intermediate storage is sometimes characterized as a form of dynamic programming, where the storage of intermediate variables reduces overall computational complexity. Moreover, this version takes advantage of analytical gradients of the subfunctions which can be evaluated to high precision more easily than depending on the finite difference formulations of gradient algorithms as in Jordan's technique.
So in summary, both a quantum implementation of classical backpropagation and Jordan's technique have a computational cost that is constant in the number of parameters if our cost model considers overall function evaluations as the cost model. This represents an exponential improvement over naive finite difference computations or symbolic evaluation of derivatives one element at a time. The backpropagation technique utilizes an extra storage register and knowledge of the problem structure, as is common in dynamic programming, while Jordan's algorithm needs only black-box queries. Both of the techniques assume bitwise access to the oracle as a classical function.
B.2 Classical parameterized Markov chains
In the previous section, the comparison of classical backpropagation and Jordan's algorithm made use of bitwise access to a classical, deterministic function. The case of a classical function encoded in bits helps frame the discussion in not only scaling but also the sense in which classical parameterized functions are perhaps not the best analog for parameterized quantum circuits. A key aspect of this difference was highlighted in Gilyén et al. [2019] by showing that in the black box setting, it was more appropriate to consider current parameterized quantum circuits as a phase or amplitude oracle, in which case they prove a lower bound of at least  $M^{1/2}$  calls to the black box (in contrast to  $O(1)$ ), ruling out the desired backpropagation scaling except for special cases. This contrast motivates asking whether the intuitive origin of this lower bound is related more to the black box nature of the access, the quantum nature of the parameterization, or merely the probabilistic features of the parameterization. Here we show that a classical analog to parameterized quantum circuits, namely parameterized Markov processes do indeed allow the analog of classical backpropagation which helps highlight that the difficulty in achieving constant scaling is due to the quantum nature of the problem.
To draw an analogy between quantum and probabilistic classical states for our purposes, we will introduce a small number of analogous concepts that are considered in greater depth by Baez and Biamonte [2012]. A parameterized quantum state  $|\psi (\theta)\rangle$  is an  $L^2$  normalized state such that
$\int_{S} ds |\psi(s; \theta)|^2 = 1$ , that is often formulated as a parameterized quantum circuit acting on a known initial state as  $|\psi(\theta)\rangle = U(\theta)|0\rangle$  where  $U$  is a unitary transformation. In contrast, a parameterized classical probability vector  $|\psi(\theta)\rangle$  is a positive  $L^1$  normalized probability vector such that  $\int_{S} ds \psi(s; \theta) = 1$ , that may be formulated as a parameterized classical circuit acting on a known reference state as  $|\psi(\theta)\rangle = U(\theta)|0\rangle$  where  $U$  is a left-stochastic operation in this case. As a connection between the two, one may consider classical transformations as the set of transformations restricted to the diagonal of a quantum density matrix, and note that it is always possible to represent a classical probability process as a quantum process, albeit non-uniquely, but the converse is of course not true in general.
The corresponding analog of expected values of Hermitian operators on quantum states will be expected values with diagonal operators  $O$ . Such operators are well defined for expected values on both classical and quantum states and are identical when the quantum populations are equal to the classical probabilities. In setting up for the computation of gradients with respect to the parameter vectors, we will consider objective functions defined by the same observable  $O$  and a sequence of operations that each depend on a single parameter. That is, the corresponding classical and quantum objectives with these assumptions may be concisely defined by
$$
f (\theta) = \int_ {S} d s O (s) \left| \left(\prod_ {i} U _ {i} \left(\theta_ {i}\right) \psi^ {0}\right) (s) \right| ^ {2} = \langle O \rangle_ {U (\theta) \psi^ {0}} \tag {18}
$$
$$
f (\theta) _ {c} = \int_ {S} d s O (s) \left(\prod_ {i} U _ {i} \left(\theta_ {i}\right) \psi_ {c} ^ {0}\right) (s) = \langle O \rangle_ {U (\theta) \psi_ {c} ^ {0}}. \tag {19}
$$
Our question here will be if the restriction to parameterized classical stochastic processes allows the desired scaling in determining gradients of an expected value with the given parameters. The evaluation of gradients with respect to parameters in quantum circuits relies largely on the fact that anti-Hermitian operators generate unitary evolutions, and we may exploit that relationship to determine gradients as expected values explicitly. There is a direct analogy to this for general stochastic operators, in that they are generated by so-called infinitesimal stochastic operators, defined by  $\sum_{i}H_{ij} = 0$ . With this definition, in finite dimensions they characterize the family of Markov semi-groups via exponentiation as  $U(t) = \exp (Ht)$ . For our purposes, it suffices that this yields a well defined operator for evaluation of single parameter derivatives.
In order to properly compare the two settings, we need to make clear a number of assumptions on the operators  $U_{i}$  and corresponding operators  $H_{i}$  that mirror assumptions in the quantum case, allowing efficient implementation. To begin, we assume each  $U_{i}(\theta_{i})$  is a simple operation, analogous to a quantum gate or Pauli operator, such that it is defined as a tensor product on a classical probabilistic bit space, and evaluating the transition probability between two basis states is efficient to do at high precision. In general, the basis could change between steps and the process could remain efficient, however for simplicity we consider the standard computational basis here. Moreover, we assume that the operation that generates the  $U_{i}$ , which we denote  $H_{i}$  is simple to evaluate between basis states, and has a bounded norm  $||H_{i}|| = 1$ , so that parameters  $\theta_{i}$  have consistent and reasonable scales. Similarly, we will restrict ourselves to observables  $O$  with reasonable norms, i.e.  $||O|| = 1$ .
With these assumptions, we investigate derivatives of a classical stochastic process under different sampling schemes. Let's imagine we have a stochastic process  $U$ , much like a variational circuit, which we write as
$$
U (\theta) = \prod_ {i} U _ {i} \left(\theta_ {i}\right) \tag {20}
$$
where each  $U_{i}$  is a stochastic process with a corresponding generator  $H_{i}$ , such that
$$
U _ {i} \left(\theta_ {i}\right) = \exp \left(\theta_ {i} H _ {i}\right) \tag {21}
$$
$$
\partial_ {\theta_ {i}} U _ {i} \left(\theta_ {i}\right) = H _ {i} U _ {i} \left(\theta_ {i}\right) \tag {22}
$$
We will be sampling the expected value of some observable  $O$  which is a diagonal matrix in our construction, and so the function value we are interested in optimizing, given a initial probability distribution  $\psi_0$  can be written in a number of ways, but some are
$$
\begin{array}{l} f (\theta) = \left\langle O \right\rangle_ {U (\theta) \psi_ {0}} (23) \\ = \int O U (\theta) \psi_ {0} (24) \\ \end{array}
$$
Now if we take the gradient of this function with respect to the parameters, we find
$$
\begin{array}{l} \partial_ {\theta_ {i}} f (\theta) = \partial_ {\theta_ {i}} \langle O \rangle_ {U (\theta) \psi_ {0}} (25) \\ = \int O \prod_ {j <   i} U _ {j} \partial_ {\theta_ {i}} U _ {i} \prod_ {k > i} U _ {k} \psi_ {0} (26) \\ = \int O \prod_ {j <   i} U _ {j} H _ {i} U _ {i} \prod_ {k > i} U _ {k} \psi_ {0}. (27) \\ \end{array}
$$
Using this construction, one can store the trajectory and lean on a path-integral formalism to use a single sampling process to take independent samples of all the gradient components with each stochastic sample that is taken. One way to write this is to borrow the path-integral like formalism using resolutions of the identity as
$$
\begin{array}{l} f (\theta) = \int O \prod_ {j} U _ {j} \psi_ {0} \tag {28} \\ = \sum_ {i _ {1}, \dots , i _ {N}} \int O \left| i _ {N}\right) \left(i _ {N} \right| U _ {N} \left| i _ {N - 1}\right) \left(i _ {N - 1} \right| U _ {N - 1} \dots \left(i _ {1} \right| \psi_ {0} \\ = \sum_ {i _ {1}, \dots , i _ {N}} p (i _ {1}, \dots , i _ {N}) O (i _ {N}) \\ \end{array}
$$