Quantum Semi-Supervised Kernel Learning

Abstract: Quantum machine learning methods have the potential to facilitate learning using extremely large datasets. While the availability of data for training machine learning models is steadily increasing, oftentimes it is much easier to collect feature vectors that to obtain the corresponding labels. One of the approaches for addressing this issue is to use semi-supervised learning, which leverages not only the labeled samples, but also unlabeled feature vectors. Here, we present a quantum machine learning algorithm for training Semi-Supervised Kernel Support Vector Machines. The algorithm uses recent advances in quantum sample-based Hamiltonian simulation to extend the existing Quantum LS-SVM algorithm to handle the semi-supervised term in the loss, while maintaining the same quantum speedup as the Quantum LS-SVM.

Reviews and author responses:

This paper proposes to extend a quantum-computing based solution of least-squares support-vector-machine to include use of unlabeled samples.  The formulation is analogous to the classical-computing case, in which semi-supervised learning introduces an additional term in the system of equations, which the authors show how to compute in the quantum setting without degrading big-O complexity.

I would lean toward rejecting this paper,  primarily on account of how the contribution relates to the publication venue.  The primary contribution lies in the procedure for preparing and propagating the quantum mechanical states needed to compute on the additional term.  Although the application is machine learning, the technique itself is still rather afar from this topic and would not appear to be of general benefit to conference-goers outside of quantum computing.  The overwhelming majority of quantum machine learning references in this paper appear in physics journals (all but one, which was ICML 2019).  Most of this paper is background material, which yet remains inadequate to convey insights into design decisions in the details of their main contribution, the derivations in section 3.2.  (I have a background in physics but not quantum computing.)   

Perhaps a paper organization more amenable to this venue would be to shift some of the lengthier equations into an appendix and use the space of the paper to discuss a more conceptual and contextual understanding of why this technique is desirable, at each step, relative to other possible quantum techniques.  For example, Figure 1 is not explained, and is not decipherable to someone outside the field, so doesn't itself add to the story.

Could be really good work, but the presentation doesn't quite come across.


EDIT:  See comments to do with paper revision, which significantly improved the presentation.

Thank you for your insightful comment! 

Indeed, quantum machine learning papers traditionally emerged from the physics community. However, we can currently observe beginnings of a trend of publishing QML papers at traditional ML conferences (the ICML paper we mentioned, a NeurIPS’19 paper on q-means by Kerenidis et al.), indicating there is an increasing emerging audience, providing potential for shifting the focus in QML to more advanced methods from the classical ML repertoire. We hope that more machine learning experts take note of recent developments in quantum computing that focus on continuous problems instead of discrete problems like search algorithms or factoring. The techniques we use in our paper are based on one such development, the introduction of quantum linear algebra tools involving density matrices. 

To make our paper more accessible to the machine learning community, we have expanded "Quantum Linear Systems of Equations" paragraph to include explanation of basics of HHL using classical linear algebra notation and the "LMR Technique for Density Operator Exponentiation" paragraph to provide more details on this fundamental technique in quantum linear algebra. We have also expanded Section 3.2 to provide more intuition behind the proposed approach for solving the semi-supervised SVM using generalized LMR technique.


Thank you for your comments and suggestions!

We have eliminated introductory material on RKHS, and instead expanded:
-  Section 2.2 paragraph "Quantum Linear Systems of Equations" to offer more insight into the HHL algorithm that underpins the original quantum SVM and our semi-supervised quantum SVM
- Section 2.2 paragraph "LMR Technique for Density Operator Exponentiation" with the method that is used in HHL to work with density matrices such as the kernel matrix
- Section 3.2, the main contribution of the manuscript.
We aimed to make these more accessible to the machine learning community.

Thank you for the review and comments!

>> Minor issues: Can any experimental study or applications be demonstrated, or only theoretical computational complexity can be compared? 

Currently, in the absence of large-scale universal quantum computers, quantum speedups for quantum machine learning algorithms are distinguished using complexity theory measures. While there are simulators such as cirq and quiskit, we have not seen them being used in quantum machine learning papers.

Based on our knowledge we introduced the first quantum semi-supervised machine learning algorithm with offering an equivalent computational complexity as quantum LS_SVM algorithm. The quantum LS-SVM in offers exponential speedup $O(\log mp)$ over the classical time complexity for solving SVM as a quadratic problem, which requires time $O(log(\epsilon^{-1})poly(p,m))$, where $\epsilon$ is the desired error.

>> 2. In Section 1.1, L is defined as L = G_I G^T_I. In this definition, whether and how the edge weights are considered? Please clarify. 

Currently, we have not considered edge weights. However, the approach we propose can be extended in a straightforward way by making the incidence matrix G_I contain nonnegative weights instead of 0/1 values, and calculating L in a similar way as calculating kernel matrix over samples, using partial trace.

The paper proposes a quantum computer-based algorithm for semi-supervised least squared kernel SVM. This work builds upon LS-SVM of Rebentrost et al (2014b) which developed a quantum algorithm for the supervised version of the problem. While the main selling point of quantum LS-SVM is that it scales logarithmically with data size, supervised algorithms shall not fully enjoy logarithmic scaling unless the cost for collecting labeled data is also logarithmic, which is unlikely. Therefore, semi-supervised setting is certainly appealing. Technically, there are two main contributions. The first is the method of providing Laplacian as an input to the quantum computer. The second contribution, which is about the computation of matrix inverse (K + KLK)^{-1}, is a bit more technical, and could be considered as the main contribution of the paper.

My main concern about the paper is on its organization. The paper provides a very gentle introduction to both semi-supervised LS SVM and quantum LS-SVM. While this helps readers to be equipped with relevant background, it is at the cost of having less space for the main contribution in Section 3.2. I would suggest to remove the content in page 2; most results about kernel methods are not really relevant to this paper. For a machine learning conference paper, one shall safely start with half-page intro in page 3. Some background in quantum computing offered in page 3, 4, 5 are quite nice, but for a conference paper, I think this is an overkill. I recommend providing the very minimal content needed to discuss Section 3.2, and then use more space to discuss the idea in 3.2 better. Specifically, Generalized LMR technique and Hermitian polynomials in Kimmel et al. (2017) could be discussed in more detail. 

This paper developes a quantum algorithm for kernel-based support vector machine working in a semi-supervised learning setting. The motivation is to utilise the significant advantage of quantum computation to train machine learning models on large-scale datasets efficiently. This paper reviews the existing work on using quantum computing for least-squares svm (via solving quantum linear systems of equations) and then extends it to deal with kernel svm in a semi-supervised setting. 

Strengths: This is an interesting emerging research topic that has its significance. Also, this paper prodives a nice tutorial on the key ideas of quantum machine learning and provides detailed derivations and analysis on the proposed algorithm.

Weaknesses: The novelty of this work seems to be incremental. It largely extends the existing algorithms such as HHL and LMR. 

Minor issues:

1. Can any experimental study or applications be demonstrated, or only theoretical comptuational complexity can be compared? 
2. In Section 1.1, L is defined as L = G_I G^T_I. In this definition, whether and how the edge weights are considered? Please clarify.   

Meta-review: Three reviewers have assessed this paper and they have scored it 6/6/6 after rebuttal with one reviewer hesitating about the appropriateness of this submission to ML venues. The reviewers have raised a number of criticisms such as an incremental nature of the paper (HHL and LMR algorithms) and the main contributions lying more within the field of quantum computing than ML. The paper was discussed with reviewers, buddy AC and chairs. On balance, it was concluded that this paper is minimally below the acceptance threshold. We encourage authors to consider all criticism, improve the paper and resubmit to another venue as there is some merit to the proposed idea.


Decision: reject