SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Di Luo *
Center for Theoretical Physics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA  
Department of Physics, Harvard University  
Cambridge, MA 02138, USA  
The NSF AI Institute for Artificial Intelligence and Fundamental Interactions  
diluo@mit.edu
Rumen Dangovski
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology Cambridge, MA 02139, USA rumenrd@mit.edu
Jiayu Shen *
Department of Physics,  
University of Illinois, Urbana-Champaign  
Urbana, IL 61801, USA  
Illinois Quantum Information Science and Technology Center  
Illinois Center for Advanced Studies of the Universe  
jiayus3@illinois.edu
Marin Soljacic
Department of Physics, Massachusetts Institute of Technology Cambridge, MA 02139, USA soljacic@mit.edu
Quantum optimization, a key application of quantum computing, has traditionally been stymied by the linearly increasing complexity of gradient calculations with an increasing number of parameters. This work bridges the gap between Koopman operator theory, which has found utility in applications because it allows for a linear representation of nonlinear dynamical systems, and natural gradient methods in quantum optimization, leading to a significant acceleration of gradient-based quantum optimization. We present Quantum-circuit Alternating Controlled Koopman learning (QuACK), a novel framework that leverages an alternating algorithm for efficient prediction of gradient dynamics on quantum computers. We demonstrate QuACK's remarkable ability to accelerate gradient-based optimization across a range of applications in quantum optimization and machine learning. In fact, our empirical studies, spanning quantum chemistry, quantum condensed matter, quantum machine learning, and noisy environments, have shown accelerations of more than 200x speedup in the overparameterized regime, 10x speedup in the smooth regime, and 3x speedup in the non-smooth regime. With QuACK, we offer a robust advancement that harnesses the advantage of gradient-based quantum optimization for practical benefits.
The dawn of quantum computing has ushered in a new era of technological advancement, presenting a paradigm shift in how we approach complex computational problems. Central to these endeavors are Variational Quantum Algorithms (VQAs) [14], which are indispensable for configuring and optimizing quantum computers. These algorithms serve as the backbone of significant domains,
*Co-first authors
including quantum optimization [54] and quantum machine learning (QML) [8]. VQAs have also influenced advances in various other quantum learning applications [17, 24, 32, 45, 28, 43, 27, 67]
At the core of VQAs lies the challenge of effectively traversing and optimizing the high-dimensional parameter landscapes that define quantum systems. To solve this, there are two primary strategies: gradient-free [72] and gradient-based methods [38]. Gradient-free methods, while straightforward and widely used in practice, do not provide any guarantees of convergence, which often results in suboptimal solutions. On the other hand, gradient-based methods, such as those employed by the Variational Quantum Eigensolver (VQE) [59, 78], offer guarantees for convergence. This characteristic has facilitated their application across a multitude of fields, such as high-energy physics [34, 65], condensed matter physics [83], quantum chemistry [58], and important optimization problems such as max-cut problem [19, 23].
More specifically, recent developments in overparameterization theory show that gradient-based methods like gradient descent provide convergence guarantees in a variety of quantum optimization tasks [37, 44, 87]. New research indicates these methods surpass gradient-free techniques such as SPSA [72] in the context of differentiable analog quantum computers [38]. Notably, the quantum natural gradient, linked theoretically with imaginary time evolution, captures crucial geometric information, thereby enhancing quantum optimization [73].
Despite these advantages, the adoption of gradient-based methods in quantum systems is not without its obstacles. The computation of gradients in these hybrid quantum-classical systems is notoriously resource-intensive and scales linearly with the number of parameters. This computational burden presents a significant hurdle, limiting the practical application of these methods on quantum computers, and thus, the full potential of quantum optimization and machine learning. Hence, we pose the question, "Can we accelerate gradient-based quantum optimization?" This is crucial to harness the theoretical advantages of convergence for practical quantum optimization tasks. In this work, we answer our question affirmatively by providing order-of-magnitude speedups to VQAs. Namely, our contributions are as follows:
- By perceiving the optimization trajectory in quantum optimization as a dynamical system, similarly to [15, 61] we bridge quantum natural gradient theory, overparameterization theory, and Koopman operator learning theory [36], which allows for a linear representation of nonlinear dynamical systems and thus is useful in applications.
- We propose Quantum-circuit Alternating Controlled Koopman Operator Learning (QuACK), a new algorithm grounded on this theory. We scrutinize its spectral stability, convex problem convergence, complexity of speedup, and non-linear extensions using sliding window and neural network methodologies.
- Through extensive experimentation in fields like quantum many-body physics, quantum chemistry, and quantum machine learning, we underscore QuACK's superior performance, achieving speedups over  $200\mathrm{x}$ ,  $10\mathrm{x}$ ,  $3\mathrm{x}$ , and  $2\mathrm{x} - 5.5\mathrm{x}$  in overparameterized, smooth, non-smooth and noisy regimes respectively.
Quantum Optimization Methods. Owing to prevailing experimental constraints, quantum optimization has frequently employed gradient-free methods such as SPSA, COBYLA, and Bayesian optimization [72, 70, 76], among others that alleviate the challenges inherent in quantum optimization [89, 25, 86]. Regarding gradient-based methods, the quantum natural gradient [73] exhibits compelling geometric properties, and conventional gradient methods such as SGD and Adam are applicable as well. Recent developments in overparameterization theory [37, 44, 87] have provided assurances for SGD's convergence in quantum optimization. We demonstrate an example where the gradient-based methods find the minimum while the gradient-free method gets trapped in Appendix C. While meta-learning techniques [81, 33, 85] have been explored to accelerate optimization across various tasks, our focus in this work is on the acceleration of gradient-based quantum optimization, given the appealing theoretical properties of such methods. We emphasize, however, that our approach is complementary to meta-learning and could be integrated as a subroutine for potential enhancements.
Some additional lines of work are related to our study. You et al. [88] analyze the convergence of quantum neural networks through the lens of the neural tangent kernel. Similarly, working through
Figure 1: QuACK: Quantum-circuit Alternating Controlled Koopman Operator Learning. (a) Parameterized quantum circuits process information; loss function is evaluated via quantum measurements. Parameter updates for the quantum circuit are computed by a classical optimizer. (b) Optimization history forms a time series, the computational cost of which is proportional to the number of parameters. (c) Koopman operator learning finds an embedding of data with approximately linear dynamics from time series in (b). (d) Koopman operator predicts parameter updates with computational cost independent of the number of parameters. Loss from predicted parameters is evaluated, and optimal parameters are used as starting point for the next iteration.
an effective quantum neural tangent kernel theory, Wang et al. [82] propose symmetric pruning to improve the loss landscape and the convergence of quantum neural networks. Finally, García-Martín et al. [20] study the effect of noise on overparameterization in quantum neural networks.
Koopman Theory. The Koopman operator theory, originating from the 1930s, furnishes a framework to understand dynamical systems [36, 80]. The theory has evolved, incorporating tools such as Dynamic Mode Decomposition (DMD) [68] (connected to Koopman mode decomposition [50, 66]) extended-DMD [84, 15, 61, 10, 3, 30, 79, 9], and machine learning approaches [48, 41, 4, 64]. While there have been successful applications in neural network optimization [16, 77, 60] and quantum mechanics [22, 35], the linking of this theory with quantum natural gradient and overparameterization theory for optimization acceleration, as we have done, is novel to our knowledge. Ours and the above contributions are built upon works on machine learning using Koopman operator theory [52, 53, 55, 71, 42].
Variational Quantum Algorithm (VQA). For a quantum mechanical system with  $N$  qubits, the key object that contains all the information of the system is called a wave function  $\psi$ . It is an  $l_{2}$ -normalized complex-valued vector in the  $2^{N}$ -dimensional Hilbert space. A parameterized quantum circuit encodes a wave function as  $\psi_{\theta}$  using a set of parameters  $\theta \in \mathbb{R}^{n_{\mathrm{params}}}$  via an ansatz layer on a quantum circuit in a quantum computer, as shown in the top-left part of Figure 1. The number of parameters,  $n_{\mathrm{params}}$ , is typically chosen to be polynomial in the number of qubits  $N$ , which scales much slower than the  $2^{N}$ -scaling of the dimension of  $\psi$  itself in the original Hilbert space.
Variational quantum eigensolver (VQE) aims to solve minimal energy wave function of a Hamiltonian with parameterized quantum circuit. A Hamiltonian  $\mathcal{H}$  describes interactions in a physical system, which mathematically is a Hermitian operator acting on the wave functions. The energy of a wave function is given by  $\mathcal{L}(\psi) = \langle \psi | \mathcal{H} \psi \rangle$ . VQE utilizes this as a loss function to find the optimal  $\theta^{*} = \arg \min_{\theta} \mathcal{L}(\theta) = \arg \min_{\theta} \langle \psi_{\theta} | \mathcal{H} \psi_{\theta} \rangle$ . Quantum machine learning follows a similar setup, aiming to minimize  $\mathcal{L}(\theta)$  involved data with parameterized quantum circuits  $\psi_{\theta}$ , which in QML are usually referred to as quantum neural networks.
To minimize the loss function, one can employ a gradient-based classical optimizer such as Adam, which requires calculating the gradient  $\partial \mathcal{L} / \partial \theta_{i}$ . In the quantum case, one usually has to explicitly evaluate the loss with a perturbation in each direction  $i$ , for example, using the parameter-shift rule [51, 69]:  $(\mathcal{L}(\theta_i + \pi /2) - \mathcal{L}(\theta_i - \pi /2)) / 2$ . This leads to a linear scaling of  $n_{\mathrm{params}}$  computational cost, making quantum optimization significantly more expensive than classical backpropagation
which computational complexity is independent of  $n_{\mathrm{params}}$ , while the required memory is still proportional to  $n_{\mathrm{params}}$ . It is worth noting that the classical computational components involved in VQE, even including training neural-network-based algorithms in the following sections, typically are much cheaper than the quantum gradient cost given the scarcity of quantum resources in practice.
Quantum Natural Gradient. The quantum natural gradient method [73] generalizes the classical natural gradient method in classical machine learning [1] by extending the concept of probability to complex-valued wave functions. It is also theoretically connected to imaginary time evolution [73]. In the context of parameter optimization, the natural gradient for updating the parameter  $\theta$  is governed by a nonlinear differential equation  $\frac{d}{dt}\pmb {\theta}(t) = -\eta F^{-1}\nabla_{\pmb{\theta}}\mathcal{L}(\pmb {\theta}(t))$ , where  $\eta$  denotes the scalar learning rate, and  $F$  represents the quantum Fisher Information matrix defined as  $F_{ij} = \langle \partial \psi_{\pmb{\theta}} / \partial \theta_{i}|\partial \psi_{\pmb{\theta}} / \partial \theta_{j}\rangle -\langle \partial \psi_{\pmb{\theta}} / \partial \theta_{i}|\psi_{\pmb{\theta}}\rangle \langle \psi_{\pmb{\theta}}|\partial \psi_{\pmb{\theta}} / \partial \theta_{j}\rangle$
Koopman Operator Learning. Consider a dynamical system characterized by a set of state variables  $x(t) \in \mathbb{R}^n$ , governed by a transition function  $T$  such that  $x(t + 1) = T(x(t))$ . According to the Koopman operator theory articulated by Koopman, a linear operator  $\mathcal{K}$  and a function  $g$  exist, satisfying  $\mathcal{K}g(x(t)) = g(T(x(t))) = g(x(t + 1))$ , where  $\mathcal{K}$  represents the Koopman operator. Generally, this operator can function in an infinite-dimensional space. However, when  $\mathcal{K}$  is restricted to a finite dimensional invariant subspace with  $g: \mathbb{R}^n \to \mathbb{R}^m$ , the Koopman operator can be depicted as a Koopman matrix  $K \in \mathbb{R}^{m \times m}$ . Data acquired from the dynamics are needed to compute the Koopman operator [12, 11]. The standard Dynamic Mode Decomposition (DMD) approach assumes  $g$  to be the identity function, predicated on the notion that the underlying dynamics of  $x$  are approximately linear, i.e.,  $T$  operates as a linear function. The extended-DMD method broadens this scope, utilizing additional feature functions such as polynomial and trigonometric functions as the basis functions for  $g$ . Further enhancing this approach, machine learning methods for the Koopman operator leverage neural networks as universal approximators for learning  $g$  [48].
4 QuACK - Quantum-circuit Alternating Controlled Koopman Operator Learning
QuACK. Our QuACK algorithm, illustrated in Algorithm 1 and Figure 1, employs a quantum circuit  $\psi_{\theta}$  with parameters  $\pmb{\theta}$  for quantum optimization or QML tasks. In Panel (a) of Figure 1 the loss function  $\mathcal{L}(\cdot)$  is stochastically evaluated through quantum measurements, and a classical optimizer updates the parameters. In Panel (b) following  $m$  gradient optimization steps, we obtain a time series of parameter updates  $\pmb{\theta}(t_0),\pmb{\theta}(t_1),\dots,\pmb{\theta}(t_m)$ . Then in Panel (c) this series is utilized by the Koopman operator learning algorithm to find an embedding for approximately linear dynamics. In Panel (d) this approach predicts the parameter updates for  $n_{\mathrm{DMD}}$  future gradient dynamics steps and calculates  $\pmb{\theta}(t_{m + 1}),\pmb{\theta}(t_{m + 2}),\dots,\pmb{\theta}(t_{m + n_{\mathrm{DMD}}})$ .
In each time step, the parameters are set directly in the quantum circuit for loss function evaluation via quantum measurements. This procedure has constant cost in terms of the number of parameters, same as the forward evaluation cost of the loss function. From the  $n_{\mathrm{DMD}}$  loss function values, we identify the lowest loss and the corresponding optimal time
Algorithm 1 QuACK   
Input: Quantum circuit  $\psi_{\theta}$  , Loss function  $\mathcal{L}(\cdot)$  , It- erations  $n_{\mathrm{iter}}$  , Koopman operator learning parame- ters m (num. simulation. steps  $n_{\mathrm{sim}}$  ), nDMD (num. DMD steps). Output: Optimal parameters  $\pmb{\theta}^{*}$    
Initialize:  $\pmb {\theta}(t_0)$  randomly   
for  $i = 0$  to  $n_{\mathrm{iter}} - 1$  do Quantum optimization or QML: for  $k = 0$  to  $m - 1$  do Compute gradient  $\nabla_{\theta}\mathcal{L}(\pmb {\theta}(t_k))$  and update  $\pmb {\theta}(t_k)\rightarrow \pmb {\theta}(t_{k + 1})$  using classical gradientbased optimizer Compute  $\mathcal{L}(\pmb {\theta}(t_{k + 1}))$    
end for Koopman operator learning: Train Koopman operator  $\kappa$  and predict optimiza- tion trajectory   
Optimal parameters selection: for  $k = m + 1$  to  $m + n_{\mathrm{DMD}}$  do Compute  $\mathcal{L}(\pmb {\theta}(t_k))$    
end for Determine  $t_\mathrm{opt}$  and set  $\pmb {\theta}(t_0)\gets \pmb {\theta}(t_\mathrm{opt})$    
end for
In this work we will interchangeably use  $m$  and  $n_{\mathrm{sim}}$  to denote the same hyperparameter. For ease of notation, we use  $m$  when indexing or looping, and  $n_{\mathrm{sim}}$  when we control for that parameter and plot dependencies.
$t_{\mathrm{opt}} = \arg \min_{t_m\leq t\leq t_{m + n_{\mathrm{DMD}}}}\mathcal{L}(\pmb {\theta}(t))$  . This step includes the last VQE iteration  $t_m$  to prevent degradation in case of inaccurate DMD predictions. The algorithm iteratively alternates the simulation steps with the Koopman steps for  $n_{\mathrm{iter}}$  steps, similarly to [77]. To facilitate the Koopman operator learning algorithm, we introduce DMD, Sliding Window DMD and neural DMD into QuACK as follows.
DMD and Sliding Window DMD. Dynamic Mode Decomposition (DMD) employs a linear fit for vector dynamics  $\pmb{\theta} \in \mathbb{R}^n$ , where  $\pmb{\theta}(t_{k+1}) = K\pmb{\theta}(t_k)$ . By concatenating  $\pmb{\theta}$  at  $m+1$  consecutive times, we define  $\Theta(t_k) \coloneqq [\pmb{\theta}(t_k)\pmb{\theta}(t_{k+1})\dots\pmb{\theta}(t_{k+m})]$ . We create data matrices  $\Theta(t_0)$  and  $\Theta(t_1)$ , the latter being the one-step evolution of the former. In approximately linear dynamics,  $K$  is constant for all  $t_k$ , and hence  $\Theta(t_1) \approx K\Theta(t_0)$ . The best fit occurs at the Frobenius loss minimum, given by  $K = \Theta(t_1)\Theta(t_0)^+$ , where  $+$  is the Moore-Penrose inverse.
When the dynamics of  $\theta$  is not linear, we can instead consider a time-delay embedding with a sliding window and concatenate the steps to form an extended data matrix [18]
$$
\boldsymbol {\Phi} (\boldsymbol {\Theta} (t _ {0})) = \left[ \begin{array}{l l l l} \phi (t _ {0}) & \phi (t _ {1}) & \dots & \phi (t _ {m}) \end{array} \right] = \left[ \begin{array}{c c c c} \boldsymbol {\theta} (t _ {0}) & \boldsymbol {\theta} (t _ {1}) & \dots & \boldsymbol {\theta} (t _ {m}) \\ \boldsymbol {\theta} (t _ {1}) & \boldsymbol {\theta} (t _ {2}) & \dots & \boldsymbol {\theta} (t _ {m + 1}) \\ \vdots & \vdots & \ddots & \vdots \\ \boldsymbol {\theta} (t _ {d}) & \boldsymbol {\theta} (t _ {d + 1}) & \dots & \boldsymbol {\theta} (t _ {m + d}) \end{array} \right]. \tag {1}
$$
$\Phi$  is generated by a sliding window of size  $d + 1$  at  $m + 1$  consecutive time steps. Each column of  $\Phi$  is a time-delay embedding for  $\Theta$ , and the different columns  $\phi$  in  $\Phi$  are embeddings at different starting times. The time-delay embedding captures some nonlinearity in the dynamics of  $\theta$ , with  $\Theta(t_{d+1}) \approx K\Phi(\Theta(t_0))$ , where  $K \in \mathbb{R}^{n \times n(d+1)}$ . The best fit is given by
$$
K = \underset {K} {\arg \min } \| \boldsymbol {\Theta} \left(t _ {d + 1}\right) - K \boldsymbol {\Phi} \left(\boldsymbol {\Theta} \left(t _ {0}\right)\right) \| _ {F} = \quad \boldsymbol {\Theta} \left(t _ {d + 1}\right) \boldsymbol {\Phi} \left(\boldsymbol {\Theta} \left(t _ {0}\right)\right) ^ {+}. \tag {2}
$$
During prediction, we start with  $\theta(t_{m+d+2}) = K\phi(t_{m+1})$  and update from  $\phi(t_{m+1})$  to  $\phi(t_{m+2})$  by removing the oldest data and adding new predicted data. This iterative prediction is performed via  $\theta(t_{k+d+1}) = K\phi(t_k)$ . Unlike the approach of Dylewsky et al. [18], we do not use an additional SVD before DMD, and our matrix  $K$  is non-square. We term this method Sliding Window DMD (SW-DMD), with standard DMD being a specific case when the sliding window size is 1 ( $d=0$ ). The time-delay embedding is related Takens' theorem [75] with similar implementations in Hankel DMD [3] and streaming DMD [26, 21].
Neural DMD. To provide a better approximation to the nonlinear dynamics, we ask whether the hard-coded sliding window transformation  $\Phi$  can be a neural network. Thus, by simply reformulating  $\Phi$  in Eq. 2 as a neural network, we formulate a natural neural objective for Koopman operator learning as  $\arg \min_{K,\alpha} \| \Theta(t_{d+1}) - K\Phi_{\alpha}(\Theta(t_0)) \|_F$ , where  $K \in \mathbb{R}^{N_{in} \times N_{out}}$  is a linear Koopman operator and  $\Phi_{\alpha}(\Theta(t_0))$  is a nonlinear neural embedding by a neural network  $\Phi_{\alpha}$  with parameters  $\alpha$ .  $\Phi_{\alpha} \coloneqq \mathrm{NN}_{\alpha} \circ \Phi$  is a composition of the neural network architecture  $\mathrm{NN}_{\alpha}$  and the sliding window embedding  $\Phi$  from the previous section.
Drawing from the advancements in machine learning for DMD, we introduce three methods: MLP-DMD, CNN-DMD, and MLP-SW-DMD. MLP-DMD uses a straightforward MLP architecture for  $\Phi$ , comprising two linear layers with an ELU activation and a residual connection, as shown in the Appendix. Unlike MLP-DMD, CNN-DMD incorporates information between simulation steps, treating these steps as the temporal dimension and parameters as the channel dimension of a 1D CNN encoder shown in the Appendix. To avoid look-ahead bias, we use causal masking of the CNN kernels, and the architecture includes two 1D-CNN layers with a bottleneck middle channel number of 1 and ELU activation to prevent overfitting. MLP-SW-DMD is similar to MLP-DMD but includes the time-delay embedding in the input, a feature we also add to CNN-DMD. As a result, MLP-DMD and CNN-DMD extend the principles of DMD, whereas MLP-SW-DMD and CNN-DMD with time-delay embedding generalize from SW-DMD. More details are available in the Appendix.
Limitations. QuACK is currently only equipped with relatively simple neural network architecture while more advanced architectures like Transformer can be explored for future work. Even though QuACK reduces the number of necessary gradient steps largely and thus achieves speedup, training
of VQA with a few gradient steps is still required to obtain the data for Koopman operator learning.  $n_{\mathrm{sim}}$  for the training gradient steps in our design is a hyperparameter (see ablation study in Appendix), and we do not have a theoretical formula for it yet.
Connection to Quantum Nature Gradient. While there exists a Koopman embedding that can linearize a given nonlinear dynamics, we provide more insights between Koopman theory and gradient-based dynamics under quantum optimization. The dynamical equation from quantum natural gradients  $d\pmb{\theta}(t) / dt = -\eta F^{-1}\nabla_{\pmb{\theta}}\mathcal{L}(\pmb{\theta}(t))$  is equivalent to  $d\psi_{\pmb{\theta}}(t) / dt = -\mathbb{P}_{\psi_{\pmb{\theta}}}\mathcal{H}\psi_{\pmb{\theta}}(t)$ , where  $F$  is the quantum Fisher information matrix, and  $\mathbb{P}_{\psi_{\pmb{\theta}}}$  represents a projector onto the manifold of the parameterized quantum circuit. When the parameterized quantum circuit possesses full expressivity, it covers the entire Hilbert space, resulting in  $d\psi_{\pmb{\theta}}(t) / dt = -\mathcal{H}\psi_{\pmb{\theta}}(t)$ . This is a linear differential equation in the vector space of unnormalized wave functions. The normalized  $\psi_{\pmb{\theta}}$  from the parameterized quantum circuit together with additional normalization factor function  $N(\pmb{\theta})$  can serve as an embedding to linearize the quantum natural gradient dynamics. For a relative short time scale, the normalized  $\psi_{\pmb{\theta}}$  already provides a good linearization which only differs from the exact dynamics by the normalization change in the short time scale. The special underlying structure of quantum natural gradient may make it easier to learn the approximate linear dynamics for Koopman operator learning.