Differentiable Analog Quantum Computing for Optimization and Control
Abstract
We formulate the ﬁrst differentiable analog quantum computing framework withspeciﬁc parameterization design at the analog signal (pulse) level to better exploitnear-term quantum devices via variational methods.
We further propose a scalableapproach to estimate the gradients of quantum dynamics using a forward passwith Monte Carlo sampling, which leads to a quantum stochastic gradient descentalgorithm for scalable gradient-based training in our framework.
Applying ourframework to quantum optimization and control, we observe a signiﬁcant advantageof differentiable analog quantum computing against SOTAs based on parameterizeddigital quantum circuits by orders of magnitude
Introduction
Quantum computing has promised unprecedented improvement in our computational ability to tackleclassically intractable problems.
Despite of the rapid development of quantum hardware [2, 62], nearterm quantum computers are still likely to have very restricted hardware resources, where the limitednumber of “qubits” and non-negligible machine noises would impede the implementation of largescale quantum algorithms.
Recent research results in both computer science [57] and physics [42]suggest a promising approach of designing resource-efﬁcient Noisy Intermediate-Scale Quantum(NISQ) [50] applications by breaking quantum circuit abstractions and directly designing applicationsat the pulse-level control of quantum machines.
The beneﬁts of this analog-oriented approach havebeen witnessed in the history of classical analog computing that predates digital computing due torelaxed hardware requirement and plays an important role in domain applications such as simulation.
One leading algorithmic paradigm on NISQ computers is the Variational Quantum Algorithm(VQA) with a few prominent examples like the Variational Quantum Eignensolver (VQE)
[49],quantum approximate optimization algorithm (QAOA)
[20], and more in [4].
Speciﬁcally, VQAuses parametrized quantum models to characterize loss functions, in particular those from quantumphysics, which are potentially intractable for classical computing [26].
These parameters willthen be optimized, usually through gradient-based approaches, to minimize the given loss function.
Conventionally, parameterized quantum models are typically quantum circuits where each gate isparameterized by classical variables.
Moreover, auto-differentiation techniques have been recentlydeveloped (e.g., [46, 56, 63]) for a scalable quantum gradient calculation on parameterized quantumcircuits.
We henceforth refer this existing framework as differentiable digital quantum computing .
Although parameterized quantum circuits are designed for NISQ applications, implementing (digital) quantum circuits still incurs non-negligible overheads, which signiﬁcantly restricts the size ofparametrized circuits that can be executed faithfully on NISQ machines.
Moreover, the current parameterization in VQAs is also largely restricted by available parameterized gates and how they compose circuits, which in turn limits the expressive power of VQAs.
A natural alternative to the current digital parameterization is to perform VQA directly on parametrized analog signals (pulses) ,either on digital quantum machines with pulse-level controls, or on general analog quantum hardware.
Indeed, parameterized analog pulses have the potential for more efﬁcient NISQ implementation andbetter expressiveness as suggested in a recent perspective paper [42], which could be a more favorableparameterization for NISQ applications even when digital quantum gates are available.
However, there is not yet a systematic study of the analog parameterization for VQAs, its scalabletraining, and its quantitative beneﬁts in achieving high-quality solutions in VQA applications.
[Table]
[Figure]
[Table]
Contributions.
We conduct the ﬁrst systematic study of the parameterization andscalable training of analog VQAs, whichwe call differentiable analog quantumcomputing .
We also leverage our scalable training to demonstrate the quantitative beneﬁts of the analog parameterization for speciﬁc VQA applications.
Speciﬁcally, we formulate the general differentiable analog quantum computing as a control problem,min v L ( v ) , where the loss function L ( v ) is calculated from the v -parametrized quantum statex ( T ; v ) generated by quantum machines at the evolution time t = T .
Intuitively, this control problemis no different from any classical ones except that the evolution of the quantum state in [0 , T ] isgoverned by the Schrödinger equation
[Equation]
(1) where the Hamiltonian operator H ( v , t ) can be much more ﬂexibly parameterized in v comparedwith parameterized quantum circuits (detailed in Section 3.2), and i is the imaginary unit.
We also develop a scalable Monte Carlo integration technique of computing quantum-related gradientsfrom the loss function
L ( v ) .
A well-known difﬁculty in computing quantum-related gradients byclassical means is the exponential cost associated with classical simulation of the quantum system.
Thus, any scalable solution must leverage quantum machines to compute the gradients for themselves.
Existing “auto-differentiation” techniques for parameterized quantum circuits (e.g., [46, 56, 63])are designed for discrete-time evolution and the basic parameterized units (i.e., gates) are describedby ﬁnite-dimensional matrices.
Differential analog quantum computing exploits parameterizationof continuous-time evolution and H ( v , t ) refers to a parameterized model from a functional space.
Our Monte Carlo integration technique is designed to bridge this technical gap, which is laterintegrated with the stochastic gradient descent (SGD) for the entire framework for fast convergenceand robustness against empirical noise.
We further illustrate that our quantum stochastic gradientdescent could still work with approximate descriptions of H ( v , t ) for domain applications.
An analogy could be drawn between our approach and classical deep learning, as summarizedin Table 1.
Neural ODEs [40, 10] view the structure of ResNet [27], x n +
= x n + f ( x n , θ ) , as thesolving of an ordinary differential equation,
[Equation]
with f ( · ) as the network layer, v the network parameter, and x ( t ) the hidden state.
This formulais similar to our system in (1), although we adopt a different parametrization than neural networks.
Similarly, differentiable physics (e.g., [31]), which incorporates physical simulation into learningprocess, leverages existing numerical solvers and the autodiff functionality of deep learning withbackpropagation to compute gradients of a physical or dynamical system, then integrates them into aneural network.
This approach has proven to accelerate learning and generalization.
Differentiable analog quantum computing can be deemed as a special form of differentiable physicsat the quantum scale.
Similar to the promise of differentiable physics or neural ODEs, we have alsoobserved the advantageous performance of differentiable analog quantum computing against theconventional parameterized quantum circuits by orders of magnitude in major VQA applicationslike optimization (Section 4) and control (Section 5).
Our auto-differentiation technique in quantumis however less efﬁcient than classical ones that leverage the chain rule and the forward/backwardpropagation.
This is due to the quantum no-cloning theorem [61] which prevents us from storingany arbitrary immediate state during quantum computation.
So, we develop a sampling technique tocompute gradients.
To sum up, the key contributions of this work include:
• A formulation of differentiable, resource-efﬁcient analog quantum computing framework (Sec. 3), 88 • A scalable technique of computing gradients by Monte Carlo sampling with SGD (Sec.
3.3),• Formal analysis on correctness, efﬁciency, and robustness that showcases exponential reduction ofcomputational complexity and bounded errors with approximate machine description (Sec. 3.4-3.6), 91 • Applications of our framework on quantum optimization (Sec. 4) and control (Sec. 5), with demonstrated advantages by orders of magnitude against parameterized quantum circuits.
Related work
Learning for dynamics and control .
Dynamical systems have widely been used to interpret andimprove the design of machine learning algorithms.
Compared to traditional discrete layers, Neural
ODEs [24] demonstrate that continuous modeling of neural network can better learn the continuousstructures [22, 54] with inﬁnite depth [44] and constant memory cost [66].
Besides neural networks,differentiable physics [16] also computes analytic gradients of classical dynamical systems like rigidbody [14], articulated body [23, 60, 52], soft body [19, 21], and ﬂuids [58, 59, 29].
Those methodshave made great success on design [41], control [28] and system identiﬁcation [38] tasks in themacroworld.
Our framework, as the ﬁrst differentiable dynamics for quantum computing, could havetremendous potential in chemistry and physics applications.
Quantum machine learning & optimal control .
Quantum machine learning is a fast-developingemerging ﬁeld (e.g., see the survey [5]) where variational quantum algorithms (VQAs) (e.g., seethe survey [4] are one of the most promising candidates for NISQ applications.
Quantum optimalcontrol (succinctly, quantum control) aims to achieve a desired response from the quantum system bycontrolling the system parameters [18, 6].
Quantum control theory has empowered the growth ofquantum technologies and found applications in several ﬁelds, ranging from molecular chemistry toquantum computing [53, 13].
The connection between quantum control theory and VQAs has beendiscussed recently [42, 45].
Several optimization algorithms have been developed to solve quantumcontrol problems, including the Krotov method [48], monotonically convergence algorithms [65],non-iterative algorithms [64], the GRadient Ascent Pulse Engineering (GRAPE) algorithm
[34],the Chopped RAndom-Basis (CRAB) algorithm [8], etc.
The development of quantum computingalso opens up the possibility of solving quantum control problems with quantum computers [17, 9].
These proposals take the approach of hybridizing quantum simulations with classical optimizationroutines.
Nevertheless, they either do not support gradient-based methods or compute the gradientsin a non-scalable way (e.g., via classical simulation), which signiﬁcantly limits their performanceespecially comparing to ours.
Existing pulse-level variational algorithms [45, 39, 15] do not discusstheir direct application to quantum analog machines and can not compute gradients analytically onquantum machines, while our paper resolves this problem.
Differentiable Analog Quantum Computing
Similar to classical dynamical systems, quantum systems also have states, governing equations,and observations, so there naturally exist plenty of optimization [47], control [12], and learning [5]problems for quantum computing.
Inspired by the success of gradient-based approach in the classicaldomain, we propose a differentiable framework to compute the gradients of parametrized analogcontrol signals on quantum computers, based on a “ forward simulation with stochastic sampling ”(see the workﬂow in Figure 1).
3.1 Quantum preliminaries
A qubit (or quantum bit ) is the analogue of a classical bit in quantum computation.
It is a two-levelquantum-mechanical system described by the Hilbert space C 2 .
We use Dirac notation | ψ (cid:105) to denotequantum states (i.e., unit vectors) ψ in H .
For example, the classical “0” and “1” are representedby | 0 (cid:105) =
(cid:20) 10 (cid:21) and | 1 (cid:105) =
(cid:20) {+
+} (cid:21) .
One qubit states could be in any linear combination of | 0 (cid:105) and | 1 (cid:105) ,which is called superposition .
An n -qubit state is a unit vector in the Kronecker tensor product ⊗ ofn single-qubit Hilbert spaces, i.e., H = ⊗ ni =1 C 2 ∼ = C 2 n , whose dimension is exponential in n .
Foran n by m matrix A and a p by q matrix B , their Kronecker product is an np by mq matrix where( A ⊗ B ) pr + u,qs + v =
A r,s B u,v .
The complex conjugate transpose of | ψ (cid:105) is denoted as (cid:104) ψ | =
| ψ (cid:105) †( † is the Hermitian conjugate).
Therefore, the inner product of φ and ψ could be written as (cid:104) φ | ψ (cid:105) .
The time evolution of quantum states under Schödinger equation is speciﬁed by the (time-dependent)
Hermitian matrix H ( t ) over the corresponding Hilbert space, known as the Hamiltonian of the
Initial State
Time Evolution
Final State
X X
+ +
Parameters:
Input
Output
Measure
MC Integration:
Differentiation of Analog Quantum Computing
Chain Rule Parameter Shift Feedback
Differentiation: ...
X
X
+ + ...... quantum system.
Typical single-qubit Hamiltonians include the famous Pauli matrices : [Equation]
[Equation]
Similarly, a multi-qubit Hamiltonian can be built from the Pauli group consisting of tensor productsof Pauli matrices.
Conventionally we write X j ( Y j , Z j ) for a multi-qubit Hamiltonian to indicate thetensor product of multiple identity matrices while the j -th operand is X ( Y , Z ), which representsoperations on the j -th subsystem.
Quantum measurement refers to the procedure of extracting classical information from quantumsystems.
It is characterized by an Hermitian matrix M called the observable .
Measuring a quantumstate | ψ (cid:105) with observable M is modelled as a random variable whose expectation value is (cid:104) ψ | M | ψ (cid:105) .
3.2 Problem formulations
Most computational tasks in quantum simulation and control, like ﬁnding the ground state of a physicssystem, can be formulated as the following optimization problem.
Given a quantum observable Mand an initial state | ψ (0) (cid:105) =
| ψ 0 (cid:105) , we seek for a parameter vector v by minimizing the loss function
[Equation]
where the evolution of | ψ ( t ) (cid:105) from t = 0 to t = T subject to the Schrödinger equation (1).
Here,
H ( v , t ) is a Hamiltonian parametrized by v with form
[Equation]
where m is the number of control pulses, H c is a time-independent Hamiltonian (e.g. Pauli matrices),
H j are tensor products of Pauli matrices, and the range of u j ( v , t ) is R .
We also require u j ( v , t ) tobe differentiable with respect to v for any t ∈ [0 , T ] .
The loss function L is hence differentiable .
We can loosen the restriction of H j to general time-independent Hamiltonians if we have powerfulenough quantum simulators.
We keep it in this paper for the convenience of presenting our algorithm.
With speciﬁc M and | ψ 0 (cid:105) , optimization problem min v L with post-processing can encode manyessential classical and quantum problems.
For example, any classical optimization of f ( x ) over n -bitintegers x can be formulated as M = (cid:80) x f ( x ) | x (cid:105) (cid:104) x
| .
More examples like Max-Cut problem andquantum state preparation are discussed in details in Sec. 4, 5.
Abstract Quantum Analog Machines (AQAMs).
We propose AQAMs as our computational model.
An AQAM optimizing the above loss function should be capable of consecutively: (1) evolving under
H ( v , t ) for any v and time interval [ t 1 , t 2 ] ⊂ [0 , T ] ; (2) evolving under constant Hamiltonian H jfor time duration τ (effectively applying unitary transformation e − iH j τ ); (3) preparing | ψ 0 (cid:105) ; (4)measuring with observable M .
For realistic quantum devices, we need to design AQAMs accordingly.
Remark 3.1.
A trivial AQAM directly employs the Hamiltonian of a quantum device as H ( v , t ) ,parameterized by tunable pulses.
In most realistic quantum devices, multi-qubit interactions are nottunable and weak compared to tunable single-qubit Hamiltonians.
Thus H j can be simulated withhigh ﬁdelity.
Our method is robust against imprecise simulations of H ( v , t ) and H j (discussed in
Section 3.6).
Hence the trivial AQAMs are usually suitable for realistic quantum devices.
Our formulation of the problem via analog quantum computing is a generalization of the formulationvia parameterized circuits.
For example, a series of parameterized Pauli rotation gates R P
j ( θ j ) =
exp {− i ( θ j / 2) P j } can be realized by H ( v , t )
= (cid:80)
j v j 1 j ( t ) P j with valuation v j = θ j / 2 , where 1 jis the indicator function of [ j, j + 1] .
However, simulating analog quantum computing via quantumcircuits requires much longer time duration on nowadays devices [42], hence is unrealistic.
So directanalog quantum computing can exploit near-term quantum devices much better than quantum circuits.
3.3 Quantum stochastic gradient descent
Classical inputs: b int , b obs (batch sizes), m (number of control pulses), u j (parametrized pulses), T (evolution duration), v (parameters)
Quantum inputs: E | ψ 0 (cid:105) (preparation of initial state),
H (parametrized system Hamiltonian), H j (pulse- controllable Hamiltonian), E M (measurement procedure with observable M ) Output: a gradient estimation of L to v for k ∈ { 1 , ..., b int } do Draw τ ∼ Uniform (0 , T ) for j ∈ { 1 , ..., m } , s ∈ {− 1 , +1 } do for l ∈ { 1 , ..., b obs } do Prepare | ψ 0 (cid:105) via E | ψ 0 (cid:105)
Evolve under H ( v , t ) for time [0 , τ ]
Evolve under H j for duration (1 + 34 s ) π .
Evolve under H ( v , t ) for time [ τ, T ] x l ← observation sample from E M .
end for ˜ p sign ( s )
j ← 1 b obs (cid:80) b obs l =1 x l end for ˜ f
k ← (cid:80) mj
=1 ∂u j ∂ v ( v , τ )(˜ p − j − ˜ p + j ) end for (cid:102) ∂
L ∂
[Equation]
Our quantum SGD scheme for computing gradients on AQAMs is illustrated in this section,with its correctness, efﬁciency, and robustnessdiscussed in the following sections.
We borrow the idea of mini-batches from classical SGD to deal with the gradients, and set twolayers of mini-batches: (1) an integration minibatch with size b int ; (2) an observation minibatch with size b obs .
The integration mini-batchupdates parameters according to the estimationof derivatives on the sampled time.
The observation mini-batch repeats experiments to generatemore precise measurement results.
The schemeis displayed in Algorithm 1.
The forward and backward propagation of our
SGD scheme is depicted in Figure 1.
Notice thatthe inner loop is the only procedure on quantum machine.
The difference of this procedurecompared with the estimation of loss function
L is the inserted evolution under H j , which isbeneﬁcial in the error analysis in Section 3.6.
With the estimation of gradients, various optimizers designed for classical stochastic gradient descentare suitable to optimize the objective function.
For example, Adam [35] is used in our experiments.
3.4 Correctness of gradient estimation
We show that Algorithm 1 generates an unbiased estimation of the gradient ∂ L ∂ v , and hence establishesits correctness.
The proof of the following theorem is postponed to Appendix B.1.
Theorem 3.2.
The derivative of L to parameters v is
[Equation]
Here, p
[Equation]
(1 ± 34 ) π U v ( t, 0) | ψ 0 (cid:105) , where
U v ( t 2 , t 1 ) denotes the time evolution operator for time interval [ t 1 , t 2 ] under Hamiltonian H ( v , t ) .
One can interpret the above formula as a direct application of the chain rule over functional derivativesδ L δu j and partial derivatives ∂u j ∂ v , since δ L δu j ( v , t ) =
p − j ( t ) − p + j ( t ) by the parameter shift rule [46, 56],which is a technique for evaluating commutators of Hermitian by quantum processes.
Algorithm 1 estimates the integral in (6) via Monte Carlo integration (MCI) technique.
We prove thatthe sampling of MCI has ﬁnite variances in Appendix B.1 when u j ( v , t ) has bounded derivatives tov .
Hence MCI converges of rate O ( b − 1 2 int ) [51].
Other numerical integral methods are also applicablehere for different conditions on u j and H j , and may have better convergence rates than MCI.
Wepresent it here because MCI has good generalization and simplicity.
We also remark that similarideas developed in this section have also appeared in [3] in the circuit model, whereas we developeverything in the analog quantum computing model.
Overall, the forward and backward propagation of differentiation of L are depicted in Figure 1.
Sincethe parameterization of u j is arbitrary, a typical treatment is using a Fourier basis or a Legendre basisas the support of the parameterization.
Neural networks are also suitable for pulse generation, whosegradients are easy to compute via backpropagation.
3.5 Scalability analysis
The resource consumption of our classical-quantum hybrid approach has two aspects: the classicaland the quantum sides.
The classical computation, as described in Algorithm 1, has O ( b int b obs m )numerical calculations.
On the quantum side, the sampling process assessing the loss functionand its derivatives takes O ( T ) time each.
The total running time on a quantum computer then is
O ( b int b obs mT ) .
Almost on every architecture of quantum devices, the number of control signals mis at most quadratic in the number of qubits n (see some survey papers [36, 32, 7, 55]), showingexcellent scalability of our approach.
We exhibit the scalability of our method for up to 11 qubitsin numerical experiments in Section 4.2.
Our approach could in principle be readily applied to themost advanced existing quantum systems (e.g., [1] with n ∼ 60 ).
This is in sharp contrast to the caseof GRAPE and CRAB algorithms, which rely on classical simulation of quantum systems with anexponential cost in n .
For instance, the associated classical computation cost for n ∼ 60 (i.e. at least2 60 ) is prohibitively high, almost reaching the limit of supercomputers today.
Our approach makes itfeasible with only O ( n 2 ) complexity .
3.6 Robustness analysis
In this section, we analyze systematic errors of gradient estimation produced of Algorithm 1.
Ourgoal is to optimize the loss function assessed on a realistic and noisy quantum machine, whosecapabilities of evolving under H ( v , t ) and H j are imperfect.
As a concrete example, when using the trivial AQAMs in Remark 3.1, the actual Hamiltonian of thequantum device may deviate from the description H ( v , t ) in our understanding, and the simulationof H j may be imprecise due to weak non-tunable terms in H c .
We show that our algorithm wellapproximates the gradient of loss function of the actual devices.
One major advantage of Algorithm 1 is that even though we have imperfect realization of Hamiltonian
H ( v , t ) built in the AQAM, the quantum part is executed on the actual quantum machine followingthe accurate Hamiltonian (cid:98) H ( v , t ) .
As a result, the output of our algorithm well approximates thegradient for the actual quantum machine.
Lemma 3.3. Let (cid:99) ∂
L ∂ v denote the accurate gradient of the loss function of the quantum machine, ∂ L ∂ vdenote the estimated gradient via Algorithm 1, and (cid:107)·(cid:107) represents the matrix spectral norm [30], then
[Equation]
The proof of this lemma is detailed in Appendix B.2.
In other words, one can use Algorithm 1to optimize the control pulses without a precise understanding of the machine Hamiltonian , if thedifference H ( v , t ) − (cid:98) H ( v , t ) varies slowly w.r.t.
v (a rather mild condition to satisfy).
On the contrary, if one relies solely on the Hamiltonian H ( v , t ) built in the abstract quantum analogmachine (e.g., the classical simulation of quantum systems in GRAPE or CRAB), the approximationerror could be as large as the difference H ( v , t ) − (cid:98) H ( v , t ) itself, a potentially large term comparedwith its derivative w.r.t.
v .
An example is illustrated in Appendix B.3.
This particular advantageof Algorithm 1 exists exactly because of its execution on the real machine, which is potentiallyapplicable in general scenarios where only approximate machine descriptions are obtainable .
Similar to the circuit case, the systematic error caused by the imprecise evolution under H j is boundedby the error sum in each evolution under H j for duration (1 ± 34 ) π , which is usually small.
Quantum optimization
Many important optimization problems in both physics and combinatorics that allow variationalsolutions can be formulated easily in our framework.
For example, ﬁnding the ground state of physics systems can be solved by variational quantum eigensolver (VQE) (e.g. [49, 33]), and searchingfor the max-cut of graphs can be approximated by quantum approximate optimization algorithms(QAOA) [20].
Replacing parameterized circuits to AQAMs in existing variational quantum algorithmsleads to signiﬁcantly improved convergence based on numerical experiments on a classical simulator.
4.1 Variational quantum eigensolver
[Figure]
We exhibit our approach via an AQAM comparable to existing circuit VQE in terms of thepulse duration, with signiﬁcantly better convergence and hardware efﬁciency in identifying theground state of the H 2 molecule.
Problem setting.
The Hamiltonian of H 2molecule is expanded with Pauli matrices in theform
[Equation]
4 Z 2 , where α i is a scalar weight calculatedin [33].
The ground state | ψ (cid:105) has the minimalenergy, deﬁned by argmin | ψ (cid:105) (cid:104) ψ | H H 2 | ψ (cid:105) .
Thisenergy function is computed on real machinesby Pauli measurements and linearity.
AQAM design.
We propose an IBM-AQAMmimicking IBM transmon system [43] (a detailed introduction in Appendix C), and realizable on IBM’s machines.
The IBM-AQAM contains 2qubits and 4 input pulses u jk ( t ) , and can evolve under
[Equation]
Here H sys is a constant Hamiltonian.
The input pulses to IBM quantum devices are u jk ( t ) whichare complex values with norm less than 1 .
These pulses are modulated by the built-in modulationprocedure M jk of the IBM’s quantum devices when executed on the real machine.
Since the tunableterms have Hamiltonian X j , we require the IBM-AQAM to be able to evolve under X j .
This isrealizable on IBM’s machines because in (7), H sys is much weaker than the microwave input pulsesfor each qubit in form X j , ensuring a high ﬁdelity simulation of Hamiltonian X j on real machines.
We also require the IBM-AQAM to support initializing in state | 00 (cid:105) and measuring with M = H H 2 ,and these procedures are easy to implement for IBM’s machines.
We adopt the parameterization
[Equation]
to make the pulse norms less than 1 , where N (0) = 0 , N ( z )
=
S ( | z | ) | z | z for z (cid:54) = 0 is a differentiablenormalization function restricting the norm, S ( x )
= 1 − e − x 1+ e − x is the shifted sigmoid function, T is theduration, and P l is the l -th Legendre polynomial.
In [33], a parameterized circuit is proposed as layered tunable single qubit rotations and ﬁxed crossresonance gates, which are compiled to pulses ﬁtting in (7) and sent to the IBM quantum devices.
Their one-layer experiments over two qubits have cross resonance gates compiled to pulses withduration around 720 dt where dt = 0 .
22 ns.
We match it in our experiments, setting T = 720 dt .
Additionally, we test our approach with only half the duration, T = 360 dt .
Comparisons.
We compare our approach to circuit VQE, ﬁnite difference method, simultaneousperturbation stochastic approximation (SPSA), and derivative-free methods (CMAES [25] and SLSQP[37]) with the IBM-AQAM on a classical simulator.
The experiment results are displayed in Figure 2awith a detailed hyper-parameter settings in Appendix D.1.1.
The circuit VQE converges to L = 0 .
02 , while our approach decreases lower: at 400 epoch, with720 dt it decreases to less than 0 .
002 , and with 360 dt it decreses to 0 .
In general, our approach isover 10 times more accurate than the circuit VQE, and 100 times more accurate than the derivative-freemethods.
The ﬁnite difference method and SPSA do not converge because of the intrinsic randomnessof quantum measurements at relatively small observation mini-batch ( b obs = 100 ), which would be ampliﬁed by the small difference length.
With a large enough observation mini-batch, the gradientsevaluated by ﬁnite difference method has ∼ 3% relative difference to the gradients evaluated by ourapproach.
These results exhibit the advantages of our differentiable analog framework compared tocircuit model and derivative-free analog models.
4.2 Quantum approximate optimization algorithm
We also investigate the application of QAOA to approximate solutions for the MaxCut problem, an
NP -complete problem.
With a corresponding AQAM, we achieve a signiﬁcantly better convergence.
Problem setting.
Given a graph G = { E, V } where V is the vertex set and E = { ( i, j ) } containsall the edges, our goal is to partition the vertices into two sets ( V 0 , V 1 ) so that the number of edgesbetween the two sets is maximized.
A cut of an n -node graph G is represented by an n -bit strings =
b 1 b 2 ...b n , with b j ∈ { 0 , 1 } representing in which set the j -th vertex is.
We use the computationalbasis | s (cid:105) in an n -qubit register to represent the cut s .
A maximum cut | s (cid:105) maximizes the expectedcut size (cid:104) s | C | s (cid:105) , where C = 12 (cid:80) (
j,k ) ∈
E C j,k , C j,k
=
I − Z j Z k .
We test the performance on thecircular graph shown in the leftmost of Figure 6.
AQAM design.
Farhi et al.
[20] optimizes a p -layer circuit
ansatz U ( β, γ )
=
Π pj =1 e − iβ j B e −
iγ j C ,where B = (cid:80) nj =1 X j .
We set p =2 as a baseline.
To have a fair comparison, we design a Cut-AQAMcorresponding to the above circuit: H ( t ) = 12 π (cid:80) 4 j =1 ( u j 0 ( t ) C j,j +1 + u j 1 ( t ) X j ) .
The input pulsesare real functions u jk ( t ) , where we restrict the energy input by requiring | u jk | ≤ 1 .
Cut-AQAMnatively supports evolving merely under C j,j +1 or X j by setting u jk as indicator functions.
We alsorequire it to support initializing in state | 0 (cid:105) and measuring with observable C .
In our experiment, weset the duration T = 4 within which the circuit QAOA can be realized by Cut-AQAM.
Similar to (8),we parameterize u jk ( v , t ) by a normalized linear combination of Legendre’s polynomial.
Comparisons.
The experiments are set up in a similar way as in Section 4.1, with details in
Appendix D.1.2.
Results are shown in Figure 2b.
The circuit QAOA and SLSQP converges to 0.08 at200 epoch, while our method converges to 2 .
6 × 10 − 6 .
Finite difference method, SPSA, and CMAESdo not converge to a value less than 1.
The analysis is similar to Section 4.1.
Since Cut-AQAMdoes not have a high frequency modulation, SLSQP also converges close to 0, but is slower thanour approach.
We conduct larger experiments for up to 11 qubits, which shows good scalability andbetter performance compared to circuit QAOA.
For details, see Figure 7 in Appendix D.1.2.
Quantum Control
Quantum control problems fall into two categories: 1) state preparation : to steer a given initial stateinto a target ﬁnal state; 2) gate synthesis : to effect a speciﬁc unitary transformation (quantum gate)in the system.
In what follows, we discuss how to formulate and solve quantum control problemsusing our differentiable analog quantum computing framework.
We demonstrate our methodology bynumerical experiments on the IBM-AQAM.
5.1 State Preparation
To prepare the target state | ψ tar (cid:105) from certain ﬁxed initial state, we desire a parameter vector v thatminimizes the loss function deﬁned in (4) with the observable M = I − | ψ tar (cid:105) [-(cid:
 )-](cid:104) ψ tar | .
We considertwo tasks: 1) to prepare the state | + (cid:105) from | 0 (cid:105) ; 2) to prepare the two-qubit maximally entangled state| Φ + (cid:105) from | 00 (cid:105) .
In both tasks, the loss function is readily computed as the measurement merelyinvolves local Pauli operators (see Appendix D.2.1), and can be carried out on the IBM-AQAM.
In the numerical experiments, the pulse duration is ﬁxed as T = 20 dt for the | + (cid:105) state, and
T = 1200 dt for the | Φ + (cid:105) state.
We also compare our method with two gradient-free methods(SLSQP, CMAES) and the GRAPE algorithm.
In both tasks, ours achieves faster convergence thanall other three methods.
In the | + (cid:105) task, the ﬁnal loss value from our method reads approximately10 − 5 , which is 18 times better than the second best result (i.e., SLSQP), as in in Figure 3 (a).
In the| Φ + (cid:105) task, the ﬁnal loss value from our method reads 0 .
034 , which is 3 times better than GRAPE and6 times better than SLSQP, as shown in Figure 3 (b).
5.2 Gate Synthesis
[Figure] Gate synthesis is more challenging than statepreparation because we hope to engineer a fullunitary gate U tar instead of just mapping a single state to a target state.
To this end we ﬁrstspecify a set of pairs S
= { ( | x j (cid:105) , | y j (cid:105) ) } kj =1that completely determines U tar in the sensethat no unitary map U other than U tar satisﬁes
[Equation]
(cid:105) [Equation] = 1 for all j = 1 , 2 , .., k .
Then,we consider the loss function L = 1 k (cid:80) kj =1 L j ,where each L j is deﬁned as in (4) with quantum observable M
=
I − | y j (cid:105) (cid:104)
y j | and initialstate | x j (cid:105) .
When L ( v ) is close to 0, the timeevolution controlled by the parameter v is approximately U tar .
The X gate and CNOT gate are widely used indigital quantum computing and supported by
IBM Qiskit.
We now exemplify our method byrecovering these two gates on the IBM-AQAM.
In both cases, the loss function can be readily computed on IBM machines.
See Appendix D.2.2 fordetail.
The pulse duration is chosen to be comparable to the built-in ones in IBM Qiskit: T = 160 dt for Xgate, and T = 1200 dt for CNOT gate.
4 We apply four methods (SLSQP, CMAES, GRAPE, ours) tothe gate synthesis tasks.
The results are shown in Figure 3 (c), (d).
In the synthesis of X gate, theﬁnal loss value from our method is 1 .
17 × 10 − 7 , which is over 10 4 times more accurate than theother three methods.
In the more involved task of synthesizing CNOT gate, our method returns apulse sequence with loss 0 .
0172 , while the results from other methods are no less than 0 .
It is worthnoting that our results also outperform the built-in gates in IBM Qiskit ( 0 . 019 for X gate and 0 .
043for CNOT gate), if the accuracy is measured using the same loss function as in our formulation.
Conclusion and Future Work
We have introduced the ﬁrst differentiable analog quantum computing framework with a quantumstochastic gradient descent algorithm that allows directly optimizing the analog pulse control signalon quantum computers.
Since the computation history in a quantum system cannot be stored or reusedfor the purpose of computing gradient, we construct a novel formulation for derivatives on quantumcomputers based on a forward pass with Monte Carlo sampling.
With the proposed algorithm,our method outperforms prior methods by orders of magnitude with better hardware efﬁciency onquantum optimization and control tasks.
Generalization.
This work suggests a larger scope of differentiable physics that extends beyondclassical cases to quantum scale, where insights and techniques from one scale could potentially betranslated to the other.
For instance, our quantum differentiable framework is largely inspired by theclassical counterpart.
We also believe some of our ﬁndings for quantum computing could also begenerally applicable in the classical paradigm: e.g., our differentiation technique is likely extendableto general linear dynamical systems and our robustness analysis also holds in general when onlyapproximate machine descriptions are known .
Limitations and Future Work.
As a ﬁrst step, we only describe our framework with a few important,but small, demonstrating examples running on simulators.
The great promise of our framework lies inimplementing large-scale VQA applications on real quantum machines and bringing useful quantum applications to practice.
To that end, we plan to incorporate real-world quantum machines into ourframework, and solve large-scale VQA tasks.
Appendix
We show the similarities between our method, Neural ODE, and differentiable physics in Figure 4.
Allthe three approaches have a differentiable system governed by some kinds of differential equations. Our method parametrizes the dynamics using continuous basis functions; Neural ODE uses neuralnetworks; and Differentiable physics describes the dynamics system using physics equations like Newton’s Second Law, Navier–Stokes equations.
[Figure]
B Theory
[Figure]
Lemma B.1. The derivative of the time evolution operator is
[Equation]
Proof. Let U v ( t 2 , t 1 ) be as deﬁned in Theorem 3.2. We can re-write (1) in the form of time evolutionoperator,i∂U
[Equation] It follows that [Equation] By the variation-of-parameters formula [11], we have
[Equation] Now, if u = v + h , we have H ( u , t ) = H ( v , t ) + h ∂H ( v ,t ) ∂ v + O ( h 2 ) . Therefore, for each t ≥ 0 , wehave
[Equation]
which implies the desired result (9) by taking h → 0 .
Proposition B.2.
Let L be deﬁned as (4) , and H ( v , t )
=
(cid:80) j f j ( v , t ) H j .
[Equation]
where | ψ ( s ) (cid:105)
=
U (0 , s ) | ψ (0) (cid:105) , M s = U † (1 , s ) MU (1 , s ) .
Proof.
The derivatives can be computed as,
[Equation]
To compute the derivative ∂ L ∂ v on a quantum machine, we invoke the parameter-shift formula.
Then we show the MCI generates an unbiased estimation of ∂ L ∂ v and converges of rate O ( b − 12 int ) .
Werequire that the derivatives of u ( v , t ) are bounded: ∀ t ∈ [0 , T ] , (cid:12)(cid:12)(cid:12) ∂u j ( [Equation] ∂ [Equation]
The integration mini-batch draws time samplings τ ∼ Uniform (0 , T ) , and evaluates
[Equation]
Then (6) turns to ∂ L ∂ v = (cid:82) T 0 f
( τ )d τ.
By MCI, the average value of T · f ( τ ) in the integrationmini-batch is an unbiased estimation of ∂ L ∂ v .
By applying the Popocivius’s inequality and the CauchySchwarz inequality, we obtain the following variance bound, which guarantees the low error of
MCI.
Proposition B.3.
The variance of f ( τ ) is ﬁnite. Speciﬁcally, Var [ f ( [Equation]
B.2 Proof of Lemma 3.3
Proof.
We let (cid:99) ∂
L ∂ v denote the accurate gradient of the loss function of the quantum machine, and ∂ L ∂ vdenote the estimated gradient via Algorithm 1.
Their analytical expressions are
[Equation]
(19) where h.c. means the Hermitian conjugate of the ﬁrst half of the expression (this is a commonabbreviation among physics, resulting in a Hermitian matrix), and
(cid:98) U v ( t 2 , t 1 ) is the evolution operatorunder
the actual Hamiltonian (cid:98) H ( v , t ) of the realistic machine.
Hence the difference between thesetwo evaluations are
[Equation]
[Equation]
where (cid:107)·(cid:107) is the spectral norm [30].
B.3 Simulating Approximated System
We show that there are cases that, when simulating the system based on imprecise approximation ofthe machine Hamiltonian, the gradient estimated is largely off to the actual gradient.
Consider a 1 qubit system H ( v, t )
=
π 4 Y + vY evolve for time in [0 , 1] with initial state | ψ 0 (cid:105) =
{+| 
+}
(cid:105) ,and measurement M =
{+| 
+}
{+(cid:105) (cid:104) 
+}
| .
Let the approximation be ˜ H ( v, t ) =
vY , whose difference to H ( v, t )is constant.
Then by Algorithm 1 where the system evolution is executed on quantum machine, theestimated gradient is exact since ∂H∂ v = ∂
˜ H∂ v =
Y.
When simulating the system with ˜ H and estimatingthe gradient as (cid:99)
∂ L ∂ v , however, consider when v = 0 .
Note
[Equation]
[Equation]
In a similar way, one can construct cases where the difference between estimated gradient based onthe approximated system and actual gradient is as large as Θ( T · (cid:107) M (cid:107) · (cid:107) H ( v, t ) − ˜ H ( v, t ) (cid:107) ) .
C IBM Pulse-level control
We adopt the following effective Hamiltonian [43] to model the pulse-level qubit control on then -qubit IBM machine:
[Equation]
The system (drift)
Hamiltonian is independent of time,
[Equation]
where E is a bidirectional connectivity graph with self loops, and J jk
= J kj . The control Hamiltonianis [Equation]
where the function M jk ( u jk , t ) modulates pulse u jk with a local oscillatory frequency ω k of qubitk :
[Equation]
27) where Ω j is the maximal energy input on qubit j , u jk ( t ) is a complex-valued control pulses and| u jk ( t ) | ≤ 1 .
On the real machine, u jk ( t ) should be a piece-wise constant complex-valued function, with eachpiece having length dt = 0 .
222 ns.
In our simulation we take u jk ( t ) as continuous function.
Takingpiece-wise constant approximation of our results will generate pulses that are available on the realmachine.
In our 1-qubit experiments, the involved constants are: n = 1 , (cid:15) 1 = 3 .
29 × 10 10 , ω 1 = 2 π × 5 . 23 ×10 9 , Ω 1 = 9 .
55 × 10 8 and E 1 = { 1 } .
In our 2-qubit experiments, the involved constants are: n = 2 , (cid:15) 1 = 3 .
{+29 × 10 10 , (cid:15) 2 = 3 . 15 ×10 10 , ω 1 = 2 π × 5 . 23 × 10 9 , ω 2 = 
 π × 5 .+}
{+01 × 10 9 , Ω 1 = 9 . 55 × 10 8 , Ω 2 = 9 . 
 × 10 8 , J 12 =1 .+}
23 × 10 7 and E 1 = E 2 = { 1 , 2 } . D Experiment Details We present the detailed experiment setups here.
Our code is written in Python and C++ and theexperiments run on a Desktop with an Intel Xeon W-2123 CPU @ 3.6GHz. In the comparisons, weuse existing packages for CMAES and SLSQP with default hyperparameters.
More details can befound in our supplementary code. D.1 Quantum Optimization [Figure]
D.1.1 Experiment details in comparison to VQE
The workﬂow of analog-ansatz-based VQE is shown in Figure 5. [Figure]
H [Figure] Modulation
[Figure] State Measure
Objective (Energy)
[Figure]
The H 2 molecule Hamiltonian at bond distance has coefﬁcients α 0 = − 1 . 0524 , α 1 = − 0 . α 2 =0 .
1809 , α 3 = − 0 . 3979 , α 4 = 0 . The parameters of different methods are presented below.
When updating, the measurements forall methods has sampling size (observation mini-batch size) b obs = 100 .
When evaluating, matrixmultiplication is utilized to have accurate loss function calculation. Hyper-parameters in our approach. The integration mini-batch size b int = 6 ; the maximal degreeof Legendre polynomial is d = 3 ; the initialization uses Gaussian distribution centered at 0 ; theoptimizer is Adam with learning rate lr = 0 .
{+02 ; time duration is T = 
 dt and T = 360 dtcorrespondingly. Circuit ansatz VQE: We use a three layered circuit ansatz. Let U j+}
( θ 1 , θ 2 , θ 3 ) = e − i θ 32 Z j e − i θ 22 X j e − i θ 12 Z j , and V j ( θ ) = U 1 ( θ j 11 , θ j 21 , θ j 31 ) U 2 ( θ j 12 , θ j 22 , θ j 32 ) . Then the circuitansatz is U ( θ ) =
V 2 ( θ ) e − i π 2 Z 1 X 2 V ( θ ) . We substitute a ZX rotation for the cross resonance gate in[33] since they generate the same entanglement. Finite difference: We use the same ansatz as in our approach.
The ﬁnite difference formula used is:
FD (cid:18)
[Equation]
(28) where e i is the unit vector whose only non-zero entry is the i -th entry with value 1 . We take a smallpertubation h = 10 − 4 and simulate the quantum system twice to evaluate corresponding inﬂuences. D.1.2 Experiment details in comparison to QAOA
The workﬂow of analog-ansatz-based QAOA is shown in Figure 6.
The settings of the measurementsand the ﬁnite difference method are similar as Appendix D.1.1.
Initial State Input Parameters Output
Final State
Measure
Objective (Cut)
[Figure]
{+Update  3  
+}
[Figure]
Hyper-parameters in our approach: The integration mini-batch size b int = 1 ; the maximal degreeof Legendre polynomial is d = 3 ; the initialization uses Gaussian distribution centered at 0 ; theoptimizer is Adam with learning rate lr = 0 .
02 ; time duration is T = 4 .
We also conduct larger experiments on cycle graph involving 6 , 9 , 11 vertices, as in Figure 7.
Ourmethod still outperforms circuit QAOA with better convergence rate and lower ﬁnal loss.
D.2 Quantum Control
D.2.1 Experiment details in state preparation
We implement our method, together with three other methods (SLSQP, CAMES, GRAPE) to the twostate preparation tasks: (a) to prepare | + (cid:105) from | 0 (cid:105) ; (b) to prepare | Φ + (cid:105) from | 00 (cid:105) .
Quantum measurement.
In both tasks, the quantum observable used in computing the loss functioncan be expressed as a sum local Pauli operators.
This nice property makes the loss function easy toevaluate on the quantum computer.
For the state | + (cid:105) , the observable is
[Equation] Similarly, we compute the observable used in preparing the | Φ + (cid:105) state: [Equation] Hyper-parameters in our approach.
The integration mini-batch size is b int = 1 in task (a), andb int = 400 in task (b). The maximal degree of Legendre polynomial is d = 4 in both cases. Theinitialization uses Gaussian distribution centered at 0 .
The optimizer is Adam in both tasks, withlearning rate lr = 0 . 01 in (a) and lr = 0 . in (b). D.2.2 Experiment details in gate synthesis
We implement our method, together with three other methods (SLSQP, CAMES, GRAPE) to the twogate synthesis tasks: (a) to synthesize X gate; (b) to synthesize CNOT gate. The matrix representationof the X gate can be found in (3). The CNOT gate is shown below:
[Equation]
Quantum measurement. We identify two sets S X and S CNOT that completely determine the X gateand CNOT gate correspondingly:
[Equation]
[Equation]
In S X and S CNOT , one of the computational basis pair can be safely removed.
But for the convenienceof training we keep them in our experiments.