Quantum Semi-Supervised Kernel Learning

Quantum machine learning methods have the potential to facilitate learning using extremely large datasets . While the availability of data for training machine learning models is steadily increasing , oftentimes it is much easier to collect feature vectors that to obtain the corresponding labels . One of the approaches for addressing this issue is to use semi-supervised learning , which leverages not only the labeled samples , but also unlabeled feature vectors . Here , we present a quantum machine learning algorithm for training Semi-Supervised Kernel Support Vector Machines . The algorithm uses recent advances in quantum sample-based Hamiltonian simulation to extend the existing Quantum LS-SVM algorithm to handle the semi-supervised term in the loss , while maintaining the same quantum speedup as the Quantum LS-SVM . 1 INTRODUCTION . Data sets used for training machine learning models are becoming increasingly large , leading to continued interest in fast methods for solving large-scale classification problems . One of the approaches being explored is training the predictive model using a quantum algorithm that has access to the training set stored in quantum-accessible memory . In parallel to research on efficient architectures for quantum memory ( Blencowe , 2010 ) , work on quantum machine learning algorithms and on quantum learning theory is under way ( see for example Refs . ( Biamonte et al. , 2017 ; Dunjko & Briegel , 2018 ; Schuld & Petruccione , 2018 ) and ( Arunachalam & de Wolf , 2017 ) for review ) . An early example of this approach is Quantum LS-SVM ( Rebentrost et al. , 2014a ) , which achieves exponential speedup compared to classical LS-SVM algorithm . Quantum LS-SVM uses quadratic least-squares loss and squared-L2 regularizer , and the optimization problem can be solved using the seminal HHL ( Harrow et al. , 2009 ) algorithm for solving quantum linear systems of equations . While progress has been made in quantum algorithms for supervised learning , it has been recently advocated that the focus should shift to unsupervised and semi-supervised setting ( Perdomo-Ortiz et al. , 2018 ) . In many domains , the most laborious part of assembling a training set is the collection of sample labels . Thus , in many scenarios , in addition to the labeled training set of size m we have access to many more feature vectors with missing labels . One way of utilizing these additional data points to improve the classification model is through semi-supervised learning . In semi-supervised learning , we are given m observations x1 , ... , xm drawn from the marginal distribution p ( x ) , where the l ( l m ) first data points come with labels y1 , ... , yl drawn from conditional distribution p ( y|x ) . Semi-supervised learning algorithms exploit the underlying distribution of the data to improve classification accuracy on unseen samples . In the approach considered here , the training samples are connected by a graph that captures their similarity . Here , we introduce a quantum algorithm for semi-supervised training of a kernel support vector machine classification model . We start with the existing Quantum LS-SVM ( Rebentrost et al. , 2014a ) , and use techniques from sample-based Hamiltonian simulation ( Kimmel et al. , 2017 ) to add a semisupervised term based on Laplacian SVM ( Melacci & Belkin , 2011 ) . As is standard in quantum machine learning ( Li et al. , 2019 ) , the algorithm accesses training points and the adjacency matrix of the graph connecting samples via a quantum oracle . We show that , with respect to the oracle , the proposed algorithm achieves the same quantum speedup as LS-SVM , that is , adding the semisupervised term does not lead to increased computational complexity . 2 PRELIMINARIES . 2.1 SEMI-SUPERVISED LEAST-SQUARES KERNEL SUPPORT VECTOR MACHINES .. Consider a problem where we are aiming to find predictors h ( x ) : X → R that are functions from a RKHS defined by a kernel K. In Semi-Supervised LS-SVMs in RKHS , we are looking for a function h ∈ H that minimizes min h∈H , b∈R γ 2 l∑ i=1 ( yi − ( h ( xi ) + b ) ) 2 + 1 2 ||h||2H + 1 2 ||∇h||2E , where γ is a user define constant allowing for adjusting the regularization strength . The last term captures the squared norm of the graph gradient on the graph G that contains all training samples as vertices , and expresses similarity between samples through , possibly edges Gu , v 1 2 ||∇h||2E = 1 2 ∑ u∼v Gu , v ( h̄u − h̄v ) 2 = h̄TLh̄ , where h̄u is the function value h ( xi ) for the vertex u corresponding to training point xi , and L is the combinatorial graph Laplacian matrix such that L [ i , j ] = Dj −Gi , j . The Representer Theorem states that if H is RKHS defined by kernel K : X × X → R , then the solution minimizing the problem above is achieved for a function h that uses only the representers of the training points , that is , a function of the form h ( x ) = ∑m j=1 cjKxj ( x ) = ∑m j=1 cjK ( xj , x ) . Thus , we can translate the problem from RKHS into a constrained quadratic optimization problem over finite , real vectors min c , ξ , b γ 2 m∑ i=1 ξ2i + 1 2 cTKc+ 1 2 cTKLKc s.t . 1− yi b+ m∑ j=1 cjK [ i , j ] = ξi where l ≤ m is the number of training points with labels ( these are grouped at the beginning of the training set ) , and h̄ = Kc , since function h is defined using representers Kxi . The semi-supervised term , the squared norm of the graph gradient of h , 1/2||∇h||2E , penalizes large changes of function h over edges of graph G. In defining the kernel K and the Laplacian L and in the two regularization terms we use all m samples . On the other hand , in calculating the empirical quadratic loss we only use the first l samples . The solution to the Semi-Supervised LS-SVMs is given by solving the following ( m+1 ) × ( m+1 ) system of linear equations [ 0 1T 1 K +KLK + γ−11 ] [ b α ] = [ 0 y ] , ( 1 ) where y = ( y1 , ... , ym ) T , 1 = ( 1 , ... , 1 ) T , 1 is identity matrix , K is kernel matrix , L is the graph Laplacian matrix , γ is a hyperparameter and α = ( α1 , ... , αm ) T is the vector of Lagrangian multipliers . 2.2 QUANTUM COMPUTING AND QUANTUM LS-SVM . Quantum computers are devices which perform computing according to the laws of quantum mechanics , a mathematical framework for describing physical theories , in language of linear algebra . Quantum Systems . Any isolated , closed quantum physical system can be fully described by a unit-norm vector in a complex Hilbert space appropriate for that system ; in quantum computing , the space is always finite-dimensional , Cd . In quantum mechanics and quantum computing , Dirac notation for linear algebra is commonly used . In Dirac notation , a vector x ∈ Cd and its complex conjugate xT , which represents a functional Cd → R , are denoted by |x〉 ( called ket ) and 〈x| ( called bra ) , respectively . We call { |ei〉 } di=1 the computational basis , where |ei〉 = ( 0 , ... , 1 , ... 0 ) T with exactly one 1 entry in the i-th position . Any |v〉 = ( v1 , ... , vd ) T can be written as |v〉 = ∑d i=1 vi|ei〉 ; coefficient vi ∈ C are called probability amplitudes . Any unit vector |x〉 ∈ Cd describes a d-level quantum state . Such a system is called a pure state . An inner product of |x1〉 , |x2〉 ∈ Cd is written as 〈x1|x2〉 . A two-level quantum state |ψ〉 = α|0〉 + β|1〉 , where |0〉 = ( 1 , 0 ) T , |1〉 = ( 0 , 1 ) T and α , β ∈ C , |α|2 + |β|2 , is called a quantum bit , or qubit for short . When both α and β are nonzero , we say |ψ〉 is in a superposition of the computational basis |0〉 and |1〉 ; the two superposition states |+〉 = 1√ 2 ( |0〉+ |1〉 ) and |−〉 = 1√ 2 ( |0〉 − |1〉 ) are very common in quantum computing . A composite quantum state of two distinct quantum systems |x1〉 ∈ Cd1 and |x2〉 ∈ Cd2 is described as tensor product of quantum states |x1〉 ⊗ |x2〉 ∈ Cd1 ⊗ Cd2 . Thus , a state of an n-qubit system is a vector in the tensor product space ( C2 ) ⊗n = C2 ⊗ C2 ⊗ ... ⊗ C2 , and is written as ∑2n−1 i=0 αi|i〉 , where i is expressed using its binary representation ; for example for n = 4 , we have |2〉 = |0010〉 = |0〉 ⊗ |0〉 ⊗ |1〉 ⊗ |0〉 . Transforming and Measuring Quantum States . Quantum operations manipulate quantum states in order to obtain some desired final state . Two types of manipulation of a quantum system are allowed by laws of physics : unitary operators and measurements . Quantum measurement , if done in the computational basis , stochastically transforms the state of the system into one of the computational basis states , based on squared magnitudes of probability amplitudes ; that is , 1√ 2 ( |0〉+ |1〉 ) will result in |0〉 and |1〉 with equal chance . Unitary operators are deterministic , invertible , normpreserving linear transforms . A unitary operator U models a transformation of a quantum state |u〉 to |v〉 = U|u〉 . Note that U|u1〉 + U|u2〉 = U ( |u1〉+ |u2〉 ) , applying a unitary to a superposition of states has the same effect as applying it separately to element of the superposition . In quantum circuit model unitary transformations are referred to as quantum gates – for example , one of the most common gates , the single-qubit Hadamard gate , is a unitary operator represented in the computational basis by the matrix H : = 1√ 2 ( 1 1 1 −1 ) . ( 2 ) Note that H|0〉 = |+〉 and H|1〉 = |−〉 . Quantum Input Model . Quantum computation typically starts from all qubits in |0〉 state . To perform computation , access to input data is needed . In quantum computing , input is typically given by a unitary operator that transforms the initial state into the desired input state for the computation – such unitaries are commonly referred to as oracles , and the computational complexity of quantum algorithms is typically measured with access to an oracle as the unit . For problems involving large amounts of input data , such as for quantum machine learning algorithms , an oracle that abstracts random access memory is often assumed . Quantum random access memory ( qRAM ) uses logN qubits to address any quantum superposition of N memory cell which may contains either quantum or classical information . For example , qRAM allows accessing classical data entries xji in quantum superposition by a transformation 1 √ mp m∑ i=1 p∑ j=1 |i , j〉|0 ... 0〉 qRAM−−−→ 1√ mp m∑ i=1 p∑ j=1 |i , j〉|xji 〉 , where |xji 〉 is a binary representation up to a given precision . Several approaches for creating quantum RAM are being considered ( Giovannetti et al. , 2008 ; Arunachalam et al. , 2015 ; Biamonte et al. , 2017 ) , but it is still an open challenge , and subtle differences in qRAM architecture may erase any gains in computational complexity of a quantum algorithm Aaronson ( 2015 ) . Quantum Linear Systems of Equations . Given an input matrix A ∈ Cn×n and a vector b ∈ Cn , the goal of linear system of equations problem is finding x ∈ Cn such that Ax = b . When A is Hermitian and full rank , the unique solution is x = A−1b . If A is not a full rank matrix then A−1 is replaced by the Moore-Penrose pseudo-inverse . HHL algorithm introduced an analogous problem in quantum setting : assuming an efficient algorithm for preparing b as a quantum state b = ∑n i=1 bi|i〉 using dlog ne+ 1 qubits , the algorithm applies quantum subroutines of phase estimation , controlled rotation , and inverse of phase estimation to obtain the state |x〉 = A −1|b〉 ‖A−1|b〉 ‖ . ( 3 ) Intuitively and at the risk of over-simplifying , HHL algorithm works as follows : if A has spectral decomposition A = ∑n i=1 λiviv T i ( where λi and vi are corresponding eigenvalues and eigenstates of A ) , then A−1 maps λivi 7→ 1 λi vi . The vector b also can be written as the linear combination of the A ’ s eigenvectors vi as b = ∑n i=1 βivi ( we are not required to compute βi ) . Then A−1b = ∑n i=1 βi 1 λi vi . In general A and A−1 are not unitary ( unless all A ’ s eigenvalues have unit magnitude ) , therefore we are not able to apply A−1 directly on |b〉 . However , since U = eiA = ∑n i=1 e iλiviv T i is unitary and has the same eigenvectors as A and A −1 , one can implement U and powers of U on a quantum computer by Hamiltonian simulation techniques ; clearly for any expected speed-up , one need to enact eiA efficiently . The HHL algorithm uses the phase estimation subroutine to estimate an approximation of λi up to a small error . The Next step computes a conditional rotation on the approximated value of λi and an auxiliary qubit |0〉 and outputs 1 λi |0〉+ √ 1− 1 λ2i |1〉 . The last step involves the inverse of phase estimation and quantum measure- ment for getting rid of garbage qubits and outputs our desired state |x〉 = A−1|b〉 = ∑n i=1 βi 1 λi vi . Density Operators . Density operator formalism is an alternative formulation for quantum mechanics that allows probabilistic mixtures of pure states , more generally referred to as mixed states . A mixed state that describes an ensemble { pi , |ψ〉i } is written as ρ = k∑ i=1 pi|ψi〉〈ψi| , ( 4 ) where ∑k i=1 pi = 1 forms a probability distribution and ρ is called density operator , which in a finite-dimensional system , in computational basis , is a semi-definite positive matrix with Tr ( ρ ) = 1 . A unitary operator U maps a quantum state expressed as a density operator ρ to UρU† , where U† is the complex conjugate of the operator U . Partial Trace of Composite Quantum System . Consider a two-part quantum system in a state described by tensor product of two density operators ρ ⊗ σ . A partial trace , tracing out the second part of the quantum system , is defined as the linear operator that leaves the first part of the system in a state Tr2 ( ρ⊗ σ ) = ρ tr ( σ ) , where Tr ( σ ) is the trace of the matrix σ . To obtain Kernel matrix K as a density matrix , quantum LS-SVM ( Rebentrost et al. , 2014b ) relies on partial trace , and on a quantum oracle that can convert , in superposition , each data point { xi } mi=1 , xi ∈ Rp to a quantum state |xi〉 = 1‖ xi ‖ ∑p t=1 ( xi ) t|t〉 , where ( xi ) t refers to the tth feature value in data point xi and assuming the oracle is given ‖xi ‖ and yi . Vector of the labels is given in the same fashion as |y〉 = 1‖ y ‖ ∑m i=1 yi|i〉 . For preparation the normalized Kernel matrix K ′ = 1 tr ( K ) K where K = XTX , we need to prepare a quantum state combining all data points in quantum superposition |X〉 = 1√∑m i=1‖ xi ‖ 2 ∑m i=1 |i〉⊗‖xi ‖ |xi〉 . The normalized Kernel matrix is obtained by discarding the training set state , K ′ = Tr2 ( |X〉〈X| ) = 1∑m i=1 ‖xi ‖ 2 m∑ i , j=1 ‖xi ‖ ‖xj ‖ 〈xi|xj〉|i〉〈j| . ( 5 ) The approach used above to construct density matrix corresponding to linear kernel matrix can be extended to polynomial kernels ( Rebentrost et al. , 2014b ) . LMR Technique for Density Operator Exponentiation . In HHL-based quantum machine learning algorithms , including in the method proposed here , matrix A for the Hamiltonian simulation within the HHL algorithm is based on data . For example , A can contain the kernel matrix K captured in the quantum system as a density matrix . Then , one need to be able to efficiently compute e−iK∆t , where K is scaled by the trace of kernel matrix . Since K is not sparse , a strategy similar to ( Lloyd et al. , 2014 ) is adapted for the exponentiation of a non-sparse density matrix : Tr1 { e−iS∆t ( K ⊗ σ ) eiS∆t } = σ − i∆t [ K , σ ] +O ( ∆t2 ) ≈ e−iK∆tσeiK∆t , ( 6 ) where S = ∑ i , j |i〉〈j| ⊗ |j〉〈i| is the swap operator and the facts Tr1 { S ( K ⊗ σ ) } = Kσ and Tr1 { ( K ⊗ σ ) S } = σK are used . The equation ( 6 ) summarizes the LMR technique : approximating e−iK∆tσeiK∆t up to error O ( ∆t2 ) is equivalent to simulating a swap operator S , applying it to the state K ⊗ σ and discarding the first system by taking partial trace operation . Since the swap operator is sparse , its simulation is efficient . Therefore the LMR trick provides an efficient way to approximate exponentiation of a non-sparse density matrix . Quantum LS-SVM . Quantum LS-SVM ( Rebentrost et al. , 2014b ) uses partial trace to construct density operator corresponding to the kernel matrixK . Once the kernel matrixK becomes available as a density operator , the quantum LS-SVM proceeds by applying the HHL algorithm for solving the system of linear equations associated with LS-LSVM , using the LMR technique for performing the density operator exponentiation e−iK∆t where the density matrix K encodes the kernel matrix .

Summary: This paper developes a quantum algorithm for kernel-based support vector machine working in a semi-supervised learning setting. The motivation is to utilise the significant advantage of quantum computation to train machine learning models on large-scale datasets efficiently. This paper reviews the existing work on using quantum computing for least-squares svm (via solving quantum linear systems of equations) and then extends it to deal with kernel svm in a semi-supervised setting.