On the Universal Approximability and Complexity Bounds of Deep Learning in Hybrid Quantum-Classical Computing

With the continuously increasing number of quantum bits in quantum computers , there are growing interests in exploring applications that can harvest the power of them . Recently , several attempts were made to implement neural networks , known to be computationally intensive , in hybrid quantum-classical scheme computing . While encouraging results are shown , two fundamental questions need to be answered : ( 1 ) whether neural networks in hybrid quantum-classical computing can leverage quantum power and meanwhile approximate any function within a given error bound , i.e. , universal approximability ; ( 2 ) how do these neural networks compare with ones on a classical computer in terms of representation power ? This work sheds light on these two questions from a theoretical perspective . 1 INTRODUCTION . Quantum computing has been rapidly evolving ( e.g. , IBM ( 2020 ) recently announced to debut quantum computer with 1,121 quantum bits ( qubits ) in 2023 ) , but the development of quantum applications is far behind ; in particular , it is still unclear what and how applications can take quantum advantages . Deep learning , one of the most prevalent applications , is well known to be computationintensive and therefore their backbone task , neural networks , is regarded as an important task to potentially take quantum advantages . Recent works ( Francesco et al. , 2019 ; Tacchino et al. , 2020 ; Jiang et al. , 2020 ) have demonstrated that the shallow neural networks with limited functions can be directly implemented on quantum computers without interfering with classical computers , but as pointed by Broughton et al . ( 2020 ) , the near-term Noisy Intermediate-Scale Quantum ( NISQ ) can hardly disentangle and generalize data in general applications , using quantum computers alone . This year , Google ( 2020 ) has put forward a library for hybrid quantum-classical neural networks , which attracts attention from both industry and academia to accelerate quantum deep learning . In a hybrid quantum-classical computing scheme , quantum computers act as hardware accelerators , working together with classical computers , to speedup the neural network computation . The incorporation of classical computers is promising to conduct operations that are hard or costly to be implemented on quantum computers ; however , it brings high data communication costs at the interface between quantum and classical computers . Therefore , instead of contiguous communication during execution , a better practice is a “ prologue-acceleration-epilogue ” scheme : the classical computer prepares data and post-processes data at prologue and epilogue , while only the quantum computer is active during the acceleration process for the main computations . Without explicit explanation , “ hybrid model ” refers to the prologue-acceleration-epilogue scheme in the rest of the paper . In a classical computing scheme , the universal approximability , i.e. , the ability to approximate a wide class of functions with arbitrary small error , and the complexity bounds of different types of neural networks have been well studied ( Cybenko , 1989 ; Hornik et al. , 1989 ; Mhaskar & Micchelli , 1992 ; Sonoda & Murata , 2017 ; Yarotsky , 2017 ; Ding et al. , 2019 ; Wang et al. , 2019 ; Fan et al. , 2020 ) . However , due to the differences in computing paradigms , not all types of neural networks can be directly implemented on quantum computers . As such , it is still unclear whether those can work with hybrid quantum-classical computing and still attain universal approximability . In addition , as quantum computing limits the types of computations to be handled , it is also unknown whether the hybrid quantum-classical neural networks can take quantum advantage over the classical networks under the same accuracy . This work explores these questions from a theoretical perspective . In this work , we first illustrate neural networks that are feasible in hybrid quantum-classical computing scheme . Then we use the method of bound-by-construction to demonstrate their universal approximability for a wide class of functions and the computation bounds , including network depth , qubit cost and gate cost , under a given error bound . In addition , compared with some of the lower complexity bounds for neural networks on classical computers , our established upper bounds are of lower asymptotic complexity , showing the potential of quantum advantage . 2 RELATED WORKS AND MOTIVATION . 2.1 NEURAL NETWORKS IN QUANTUM COMPUTING . Although the research on neural networks in quantum computing can trace back to the 1990s ( Kak , 1995 ; Purushothaman & Karayiannis , 1997 ; Ezhov & Ventura , 2000 ) , but only recently , along with the revolution of quantum computers , the implementation of neural networks on actual quantum computer emerges ( Francesco et al. , 2019 ; Jiang et al. , 2020 ; Bisarya et al. , 2020 ) . There are mainly three different directions to exploit the power of quantum computers : ( 1 ) applying the Quantum Random Access Memory ( QRAM ) ( Blencowe , 2010 ) ; ( 2 ) employing pure quantum computers ; ( 3 ) bridging different platforms for a hybrid quantum-classical computing ( McClean et al. , 2016 ) . Kerenidis et al . ( 2019 ) is a typical work to implement neural networks with QRAM . Using QRAM provides the highest flexibility , such as implementing non-linear functions using lookup tables . But QRAM itself has limitations : instead of using the widely applied superconducting qubits ( Arute et al. , 2019 ; IBM , 2016 ) , QRAM needs the support of spin qubit ( Veldhorst et al. , 2015 ) to provide relatively long lifetime . To make the system practical , there is still a long way to go . Alternatively , there are works which encode data to either qubits ( Francesco et al. , 2019 ) or qubit states ( Jiang et al. , 2020 ) and use superconducting-based quantum computers to run neural networks . These methods also have limitations : Due to the short decoherence times in current quantum computers , the condition statement is not supported , making it hard to implement some non-linear functions such as the most commonly used Rectified Linear Unit ( ReLU ) . But the advantages are also obvious : ( 1 ) the designs can be directly evaluated on actual quantum computers ; ( 2 ) little communication is needed between quantum and classical computers , which may otherwise be expensive . Hybrid quantum-classical computing tries to address the limitations of QRAM and pure quantum computer based approaches . Broughton et al . ( 2020 ) establishes a computing paradigm where different neurons can be implemented on either quantum or classical computers . This brings the flexibility in implementing functions ( e.g. , ReLU ) , while at the same time , it calls for fast interfaces for massive data transfer between quantum and classical computers . In this work , we focus on the hybrid quantum-classical computing scheme and follow the “ prologueacceleration-epilogue ” computing scheme . It offers the flexibility of implementation and at the same time requires minimal quantum-classical data transfer , as demonstrated in Figure 2 . 2.2 UNIVERSAL APPROXIMATION AND COMPLEXITY BOUND . Universal approximability of neural network indicates that for any given continuous function or a wide class of functions satisfying some constraints , and arbitrarily small error bound > 0 , there exists a neural network model which can approximate the function with no more than error . On classical computing , different types of neural networks have been proved to have universal approximability : multi-layer feedforward neural networks ( Cybenko , 1989 ; Hornik et al. , 1989 ) ; ReLU neural networks ( Mhaskar & Micchelli , 1992 ; Sonoda & Murata , 2017 ; Yarotsky , 2017 ) ; quantized neural networks ( Ding et al. , 2019 ; Wang et al. , 2019 ) ; and quadratic neural networks ( Fan et al. , 2020 ) . In addition , many of these works also establishes complexity bounds in terms of the number of weights , number of layers , or number of neurons needed for approximation with error bound . When it comes to quantum computing , in recent years , Delgado ( 2018 ) demonstrated quantum circuit with an additional trainable diagonal matrix can approximate the given functions , and Schuld et al . ( 2020 ) shown that the Fourier-type sum-based quantum models can be universal function approximators if the quantum circuit is measured enough many times . Most recently , we are wit- nessing the exponentially increasing research works to exploit the high-parallelism provided by quantum computers to accelerate neural networks ( Perdomo-Ortiz et al. , 2018 ; Huggins et al. , 2019 ; Cong et al. , 2019 ; Kerenidis et al. , 2019 ; Francesco et al. , 2019 ; Bisarya et al. , 2020 ; Broughton et al. , 2020 ; Jiang et al. , 2020 ; Tacchino et al. , 2020 ; Xia & Kais , 2020 ) . The existing works have demonstrated that the quantum neural network can achieve state-of-the-art accuracy for some tasks . For example , Figure 1 demonstrates the test accuracy comparison of different neural networks targeting pairwise classifiers in MNIST ( LeCun et al. , 1998 ) ) . The average accuracy gap between that in Figure 1 ( a ) and Figure 1 ( c ) is merely 0.5 % . However , the penalty for achieving high-parallelism in quantum computing is the constrained computation types to be performed . As a result , neural networks designed for quantum computing has limited operations , indicating that the networks designed for classical computing may not be implemented on quantum computers . This raises a fundamental problem : whether the neural networks in quantum computing can attain universal approximability ? The failure to attain universal approximability will fundamentally hinder the neural networks in quantum computing being used in practice due to the significant accuracy loss for specific functions . Therefore , it is imminent to understand the expressivity of neural networks dedicated to quantum computers . Motivated by this , we prove that the neural networks in a hybrid quantum-classical computing can approximate a wide class of functions with arbitarily small error . We also establish complexity bounds , which gives practical insights in designing networks for quantum computing . 3 MAIN RESULTS . Figure 2 illustrates the adopted prologue-acceleration-epilogue computing scheme . It is a practical scheme with the small number of quantum-classical data transfer and the neural network designed for this scheme can achieve competitive accuracy against classical computing as demonstrated in Figure 1 . In addition , the target computing scheme is a special case of that used in Tensorflow Quantum ( Broughton et al. , 2020 ) ; therefore , if we can prove that the neural networks designed for it have universal approximability , then the conclusion can be directly derived to Tensorflow Quantum . In this work , we follow the idea from ( Yarotsky , 2017 ; Ding et al. , 2019 ) by constructing a neural network ( namely BPNN , see Section 4.1 ) in the prologue-acceleration-epilogue computing scheme ( see Section 4.4 ) with bounded maximum error ( see Section 4.3 ) for a wide class of functions ( denoted as Fd , n , see Section 4.2 ) . In such a proof , the fundamental function to be implemented is the Taylor polynomial . In the next , we first state the main result of the Taylor polynomial of a function f ∈ Fd , n on the quantum computing ( see Appendix A.5 for the formal version ) . Results 3.1 . For any given function f ∈ Fd , n and an expansion point k , its Taylor polynomial at point k can be implemented on the quantum computer , such that ( i ) the network can exactly implements the Taylor polynomial , ( ii ) the depth is O ( log n ) , ( iii ) the number of gates is O ( n2 log n log d ) , ( iv ) the number of qubits is O ( n2 log d ) . Here , we observe that since the Taylor function can be exactly implemented by the quantum computer , the complexities on depth , gates , and qubits are not related to error bound . This will be the root cause that the upper bound of neural networks in hybrid quantum-classical computing scheme can approach to the lower bound of ones on classical computing ( see the comparison at the end of this section ) . In addition , the classical computing can not build such a system , because there are too many inputs , reaching up to dn+1 , which is infeasible for classical computing with exponentially increasing inputs . On the other hand , quantum computing can take advantage of encoding n inputs to log n qubits , and therefore , it is feasible to implement such a network on a quantum computer . The above result shows the ability of BPNN to exactly implement Taylor expansion at any point . Then , combined with the classical Prologue for quantum-state preparation and the classical Epilogue for accumulate results at all expansion points , we next state the main result of approximating a function f ∈ Fd , n on the prologue-acceleration-epilogue computing scheme as follows ; the formal version can be found in Appendix A.6 . Results 3.2 . For any given function f ∈ Fd , n , there is a binary polynomial neural network with a fixed structure that can be implemented in the hybrid quantum-classical computing scheme , such that ( i ) the network can approximate f with any error ∈ ( 0 , 1 ) , ( ii ) the overall depth is O ( 1 ) ; ( iii ) the number of quantum gates is O ( ( 1/ ) d n ) ; ( iv ) the number of qubits is O ( ( 1/ ) d n ) ; ( v ) the number of weights on classical computer is O ( ( 1/ ) d n ) . From the above result , we can see that the upper bounds on the depth and the number of weight/gates are of the same asymptotic complexity for both the quantum portion and classical portion in the hybrid computing system , which satisfies the constraint discussed in Section 4.1 to take full use of quantum computing . We further compare the complexity bounds between the BPNN on the hybrid quantum-classical computing scheme against the classical networks constructed for bound analysis . Comparison with the upper bounds for neural networks on classical computers : To attain an approximation error , Fan et al . ( 2020 ) demonstrates that the upper bound on the number of weights for unquantized quadratic network is O ( log ( log ( 1/ ) ) × ( 1/ ) d n ) ) , and Ding et al . ( 2019 ) demonstrates that the upper bound on the number of binary weights of the ReLU neural network is O ( log2 ( 1/ ) × ( 1/ ) dn ) . On the other hand , for the BPNN on hybrid quantum-classical computing , both the number of gates used in quantum acceleration and the weights used in classical prologue and epilogue are O ( ( 1/ ) d n ) . Although BPNN has similar expressive power compared with the binary ReLU network and reduced expressive power compared with the unquantized quadratic network ( due to the constraints on weight selection ) , the obtained upper bounds are of asymptotically lower complexity , which again shows the benefits of quantum computing for neural networks . Comparison with the lower bounds for neural networks on classical computers : We further compare the lower bound of the number of weights/gates needed to attain an error bound on a classical computer . The only established result in the literature is for unquantized ReLU network ( Yarotsky , 2017 ) , which suggests that to attain an approximation error bound of , the number of weights needed is at least Ω ( log−2p−1 ( 1/ ) × ( 1/ ) d/n ) with depth constraint of O ( logp ( 1/ ) ) where p is a constant to be chosen to determine the growth rate of depth . In this work , we demonstrate that the depth of BPNN in hybrid quantum-classical computing can be O ( 1 ) and the upper bounds on the number of weight/gates are O ( ( 1/ ) d/n ) ( both quantum and classical computers ) . Apparently , our upper bounds are even approching to the lower bound of the networks on classical computers , which are unquantized and should have stronger expressive power . This clearly demonstrates the potential quantum advantage that can be attained .

Summary: The paper considers the expressivity and approximation properties of machine learning models where a parameterized quantum circuit is used to 'accelerate' a classical neural network. The results consider a model with a date encoder, a quantum circuit, and then a classical feedforward neural net for post-processing. To make the results non-trivial only models with asymptotically similar classical and quantum complexity are considered.