On the Universal Approximability and Complexity Bounds of Deep Learning in Hybrid Quantum-Classical Computing

With the continuously increasing number of quantum bits in quantum computers , there are growing interests in exploring applications that can harvest the power of them . Recently , several attempts were made to implement neural networks , known to be computationally intensive , in hybrid quantum-classical scheme computing . While encouraging results are shown , two fundamental questions need to be answered : ( 1 ) whether neural networks in hybrid quantum-classical computing can leverage quantum power and meanwhile approximate any function within a given error bound , i.e. , universal approximability ; ( 2 ) how do these neural networks compare with ones on a classical computer in terms of representation power ? This work sheds light on these two questions from a theoretical perspective . 1 INTRODUCTION . Quantum computing has been rapidly evolving ( e.g. , IBM ( 2020 ) recently announced to debut quantum computer with 1,121 quantum bits ( qubits ) in 2023 ) , but the development of quantum applications is far behind ; in particular , it is still unclear what and how applications can take quantum advantages . Deep learning , one of the most prevalent applications , is well known to be computationintensive and therefore their backbone task , neural networks , is regarded as an important task to potentially take quantum advantages . Recent works ( Francesco et al. , 2019 ; Tacchino et al. , 2020 ; Jiang et al. , 2020 ) have demonstrated that the shallow neural networks with limited functions can be directly implemented on quantum computers without interfering with classical computers , but as pointed by Broughton et al . ( 2020 ) , the near-term Noisy Intermediate-Scale Quantum ( NISQ ) can hardly disentangle and generalize data in general applications , using quantum computers alone . This year , Google ( 2020 ) has put forward a library for hybrid quantum-classical neural networks , which attracts attention from both industry and academia to accelerate quantum deep learning . In a hybrid quantum-classical computing scheme , quantum computers act as hardware accelerators , working together with classical computers , to speedup the neural network computation . The incorporation of classical computers is promising to conduct operations that are hard or costly to be implemented on quantum computers ; however , it brings high data communication costs at the interface between quantum and classical computers . Therefore , instead of contiguous communication during execution , a better practice is a “ prologue-acceleration-epilogue ” scheme : the classical computer prepares data and post-processes data at prologue and epilogue , while only the quantum computer is active during the acceleration process for the main computations . Without explicit explanation , “ hybrid model ” refers to the prologue-acceleration-epilogue scheme in the rest of the paper . In a classical computing scheme , the universal approximability , i.e. , the ability to approximate a wide class of functions with arbitrary small error , and the complexity bounds of different types of neural networks have been well studied ( Cybenko , 1989 ; Hornik et al. , 1989 ; Mhaskar & Micchelli , 1992 ; Sonoda & Murata , 2017 ; Yarotsky , 2017 ; Ding et al. , 2019 ; Wang et al. , 2019 ; Fan et al. , 2020 ) . However , due to the differences in computing paradigms , not all types of neural networks can be directly implemented on quantum computers . As such , it is still unclear whether those can work with hybrid quantum-classical computing and still attain universal approximability . In addition , as quantum computing limits the types of computations to be handled , it is also unknown whether the hybrid quantum-classical neural networks can take quantum advantage over the classical networks under the same accuracy . This work explores these questions from a theoretical perspective . In this work , we first illustrate neural networks that are feasible in hybrid quantum-classical computing scheme . Then we use the method of bound-by-construction to demonstrate their universal approximability for a wide class of functions and the computation bounds , including network depth , qubit cost and gate cost , under a given error bound . In addition , compared with some of the lower complexity bounds for neural networks on classical computers , our established upper bounds are of lower asymptotic complexity , showing the potential of quantum advantage . 2 RELATED WORKS AND MOTIVATION . 2.1 NEURAL NETWORKS IN QUANTUM COMPUTING . Although the research on neural networks in quantum computing can trace back to the 1990s ( Kak , 1995 ; Purushothaman & Karayiannis , 1997 ; Ezhov & Ventura , 2000 ) , but only recently , along with the revolution of quantum computers , the implementation of neural networks on actual quantum computer emerges ( Francesco et al. , 2019 ; Jiang et al. , 2020 ; Bisarya et al. , 2020 ) . There are mainly three different directions to exploit the power of quantum computers : ( 1 ) applying the Quantum Random Access Memory ( QRAM ) ( Blencowe , 2010 ) ; ( 2 ) employing pure quantum computers ; ( 3 ) bridging different platforms for a hybrid quantum-classical computing ( McClean et al. , 2016 ) . Kerenidis et al . ( 2019 ) is a typical work to implement neural networks with QRAM . Using QRAM provides the highest flexibility , such as implementing non-linear functions using lookup tables . But QRAM itself has limitations : instead of using the widely applied superconducting qubits ( Arute et al. , 2019 ; IBM , 2016 ) , QRAM needs the support of spin qubit ( Veldhorst et al. , 2015 ) to provide relatively long lifetime . To make the system practical , there is still a long way to go . Alternatively , there are works which encode data to either qubits ( Francesco et al. , 2019 ) or qubit states ( Jiang et al. , 2020 ) and use superconducting-based quantum computers to run neural networks . These methods also have limitations : Due to the short decoherence times in current quantum computers , the condition statement is not supported , making it hard to implement some non-linear functions such as the most commonly used Rectified Linear Unit ( ReLU ) . But the advantages are also obvious : ( 1 ) the designs can be directly evaluated on actual quantum computers ; ( 2 ) little communication is needed between quantum and classical computers , which may otherwise be expensive . Hybrid quantum-classical computing tries to address the limitations of QRAM and pure quantum computer based approaches . Broughton et al . ( 2020 ) establishes a computing paradigm where different neurons can be implemented on either quantum or classical computers . This brings the flexibility in implementing functions ( e.g. , ReLU ) , while at the same time , it calls for fast interfaces for massive data transfer between quantum and classical computers . In this work , we focus on the hybrid quantum-classical computing scheme and follow the “ prologueacceleration-epilogue ” computing scheme . It offers the flexibility of implementation and at the same time requires minimal quantum-classical data transfer , as demonstrated in Figure 2 . 2.2 UNIVERSAL APPROXIMATION AND COMPLEXITY BOUND . Universal approximability of neural network indicates that for any given continuous function or a wide class of functions satisfying some constraints , and arbitrarily small error bound > 0 , there exists a neural network model which can approximate the function with no more than error . On classical computing , different types of neural networks have been proved to have universal approximability : multi-layer feedforward neural networks ( Cybenko , 1989 ; Hornik et al. , 1989 ) ; ReLU neural networks ( Mhaskar & Micchelli , 1992 ; Sonoda & Murata , 2017 ; Yarotsky , 2017 ) ; quantized neural networks ( Ding et al. , 2019 ; Wang et al. , 2019 ) ; and quadratic neural networks ( Fan et al. , 2020 ) . In addition , many of these works also establishes complexity bounds in terms of the number of weights , number of layers , or number of neurons needed for approximation with error bound . When it comes to quantum computing , in recent years , Delgado ( 2018 ) demonstrated quantum circuit with an additional trainable diagonal matrix can approximate the given functions , and Schuld et al . ( 2020 ) shown that the Fourier-type sum-based quantum models can be universal function approximators if the quantum circuit is measured enough many times . Most recently , we are wit- nessing the exponentially increasing research works to exploit the high-parallelism provided by quantum computers to accelerate neural networks ( Perdomo-Ortiz et al. , 2018 ; Huggins et al. , 2019 ; Cong et al. , 2019 ; Kerenidis et al. , 2019 ; Francesco et al. , 2019 ; Bisarya et al. , 2020 ; Broughton et al. , 2020 ; Jiang et al. , 2020 ; Tacchino et al. , 2020 ; Xia & Kais , 2020 ) . The existing works have demonstrated that the quantum neural network can achieve state-of-the-art accuracy for some tasks . For example , Figure 1 demonstrates the test accuracy comparison of different neural networks targeting pairwise classifiers in MNIST ( LeCun et al. , 1998 ) ) . The average accuracy gap between that in Figure 1 ( a ) and Figure 1 ( c ) is merely 0.5 % . However , the penalty for achieving high-parallelism in quantum computing is the constrained computation types to be performed . As a result , neural networks designed for quantum computing has limited operations , indicating that the networks designed for classical computing may not be implemented on quantum computers . This raises a fundamental problem : whether the neural networks in quantum computing can attain universal approximability ? The failure to attain universal approximability will fundamentally hinder the neural networks in quantum computing being used in practice due to the significant accuracy loss for specific functions . Therefore , it is imminent to understand the expressivity of neural networks dedicated to quantum computers . Motivated by this , we prove that the neural networks in a hybrid quantum-classical computing can approximate a wide class of functions with arbitarily small error . We also establish complexity bounds , which gives practical insights in designing networks for quantum computing . 3 MAIN RESULTS . Figure 2 illustrates the adopted prologue-acceleration-epilogue computing scheme . It is a practical scheme with the small number of quantum-classical data transfer and the neural network designed for this scheme can achieve competitive accuracy against classical computing as demonstrated in Figure 1 . In addition , the target computing scheme is a special case of that used in Tensorflow Quantum ( Broughton et al. , 2020 ) ; therefore , if we can prove that the neural networks designed for it have universal approximability , then the conclusion can be directly derived to Tensorflow Quantum . In this work , we follow the idea from ( Yarotsky , 2017 ; Ding et al. , 2019 ) by constructing a neural network ( namely BPNN , see Section 4.1 ) in the prologue-acceleration-epilogue computing scheme ( see Section 4.4 ) with bounded maximum error ( see Section 4.3 ) for a wide class of functions ( denoted as Fd , n , see Section 4.2 ) . In such a proof , the fundamental function to be implemented is the Taylor polynomial . In the next , we first state the main result of the Taylor polynomial of a function f ∈ Fd , n on the quantum computing ( see Appendix A.5 for the formal version ) . Results 3.1 . For any given function f ∈ Fd , n and an expansion point k , its Taylor polynomial at point k can be implemented on the quantum computer , such that ( i ) the network can exactly implements the Taylor polynomial , ( ii ) the depth is O ( log n ) , ( iii ) the number of gates is O ( n2 log n log d ) , ( iv ) the number of qubits is O ( n2 log d ) . Here , we observe that since the Taylor function can be exactly implemented by the quantum computer , the complexities on depth , gates , and qubits are not related to error bound . This will be the root cause that the upper bound of neural networks in hybrid quantum-classical computing scheme can approach to the lower bound of ones on classical computing ( see the comparison at the end of this section ) . In addition , the classical computing can not build such a system , because there are too many inputs , reaching up to dn+1 , which is infeasible for classical computing with exponentially increasing inputs . On the other hand , quantum computing can take advantage of encoding n inputs to log n qubits , and therefore , it is feasible to implement such a network on a quantum computer . The above result shows the ability of BPNN to exactly implement Taylor expansion at any point . Then , combined with the classical Prologue for quantum-state preparation and the classical Epilogue for accumulate results at all expansion points , we next state the main result of approximating a function f ∈ Fd , n on the prologue-acceleration-epilogue computing scheme as follows ; the formal version can be found in Appendix A.6 . Results 3.2 . For any given function f ∈ Fd , n , there is a binary polynomial neural network with a fixed structure that can be implemented in the hybrid quantum-classical computing scheme , such that ( i ) the network can approximate f with any error ∈ ( 0 , 1 ) , ( ii ) the overall depth is O ( 1 ) ; ( iii ) the number of quantum gates is O ( ( 1/ ) d n ) ; ( iv ) the number of qubits is O ( ( 1/ ) d n ) ; ( v ) the number of weights on classical computer is O ( ( 1/ ) d n ) . From the above result , we can see that the upper bounds on the depth and the number of weight/gates are of the same asymptotic complexity for both the quantum portion and classical portion in the hybrid computing system , which satisfies the constraint discussed in Section 4.1 to take full use of quantum computing . We further compare the complexity bounds between the BPNN on the hybrid quantum-classical computing scheme against the classical networks constructed for bound analysis . Comparison with the upper bounds for neural networks on classical computers : To attain an approximation error , Fan et al . ( 2020 ) demonstrates that the upper bound on the number of weights for unquantized quadratic network is O ( log ( log ( 1/ ) ) × ( 1/ ) d n ) ) , and Ding et al . ( 2019 ) demonstrates that the upper bound on the number of binary weights of the ReLU neural network is O ( log2 ( 1/ ) × ( 1/ ) dn ) . On the other hand , for the BPNN on hybrid quantum-classical computing , both the number of gates used in quantum acceleration and the weights used in classical prologue and epilogue are O ( ( 1/ ) d n ) . Although BPNN has similar expressive power compared with the binary ReLU network and reduced expressive power compared with the unquantized quadratic network ( due to the constraints on weight selection ) , the obtained upper bounds are of asymptotically lower complexity , which again shows the benefits of quantum computing for neural networks . Comparison with the lower bounds for neural networks on classical computers : We further compare the lower bound of the number of weights/gates needed to attain an error bound on a classical computer . The only established result in the literature is for unquantized ReLU network ( Yarotsky , 2017 ) , which suggests that to attain an approximation error bound of , the number of weights needed is at least Ω ( log−2p−1 ( 1/ ) × ( 1/ ) d/n ) with depth constraint of O ( logp ( 1/ ) ) where p is a constant to be chosen to determine the growth rate of depth . In this work , we demonstrate that the depth of BPNN in hybrid quantum-classical computing can be O ( 1 ) and the upper bounds on the number of weight/gates are O ( ( 1/ ) d/n ) ( both quantum and classical computers ) . Apparently , our upper bounds are even approching to the lower bound of the networks on classical computers , which are unquantized and should have stronger expressive power . This clearly demonstrates the potential quantum advantage that can be attained .

Summary: The problem studied in this work is of interest in the quantum machine learning community, as the power of small and noisy quantum computers for machine learning problems is far from being understood. Therefore, it is important to study the expressivity of quantum neural networks as function approximators. This work uses the model introduced by Tensorflow Quantum, where different neurons can be implemented on either quantum or classical computers.