Paper: Variational Quantum Linear Solver enhanced Quantum Support Vector machine (ICLR-2024)

Reviewer comment (Novelty):
The proposed method is too simple and outdated. The essence of this paper is utilizing HEA to solve classification problems which are widely studied in the previous literature [1,2]. Using HEA to solve such a problem is quite trivial and there seems no supremacy in using HEA to solve this kind of problem. There is also literature using Quantum Architecture Search to design problem-specific ansatz for classification problems [3,4].

Author response:
**To Point One:**
The performance and application of HEA have been extensively discussed in these literature. I concur with these findings.

In our research, we noticed that VQLS  demonstrates notable scalability on NISQ- devices for handling larger problem sizes with toy examples, as detailed in [1]. After a thorough review of papers citing VQLS's original research, a key gap we observed was the lack of exploration into VQLS's performance in real-world scenarios. This paper aims to bridge this gap, exploring an area previously uncharted in quantum computing research.

By utilizing HEA into our algorithm, we have effectively tackled implementation challenges, demonstrating the practicality and feasibility of QSVM in the context of NISQ technology. Our results provide valuable insights into the practical applications and scalability of VQLS, offering a new perspective to those interested in the field.

While it appears that HEA doesn't necessarily offer superiority in solving such problems, our research underscores the importance of continual exploration and innovation. Each new study, including ours, brings us closer to realizing the full potential of VQLS theoretically and numerically.

Reviewer comment (Theory):
This paper lacks a theoretical analysis of the expressivity and the trainability of the proposed quantum ansatz.

Author response:
**To Point Two:**

Having established the feasibility of our algorithm, we plan to employ these two descriptors to evaluate the performance of various Ansätze. Your insightful comments have inspired us to further investigate how these descriptors impact the performance of our algorithm across different Ansätze.

Reviewer comment (Experiments):
The tailored Iris dataset is too simple. The authors only conduct experiments with 3 qubits and a very small kernel, which seems no big difference from the toy dataset. The authors constantly mention that the main advancement of this paper is that they use real-world datasets, but results at this level are clearly not enough (with much larger scaling on MNIST [2]).

Author response:
**To Point Three:**

Our approach differs notably from the use of toy datasets, which often select specific cases to demonstrate their concepts.

The HHL-based QSVM was initially implemented as described in [2]. However, its kernel  must satisfy this limitation: the condition number of the kernel matrix needs to be scaled to polylog(N), where N represents the size of the linear system, as detailed in [3]. In contrast, our experiments involved the random selection of seven training samples applied to three qubits.  This critical distinction that we believe is important to highlight.

Furthermore, after having submitted our paper, *we expanded our evaluation to include additional datasets concerning heart disease and TIG aluminium 5083 from Kaggle, also implemented on three qubits. When these results affirmatively demonstrate the performance of our approach, we are ready to update our paper with these findings upon receiving your feedback.*

As for the scalability of our algorithm, we are currently reviewing literature on it and strategies to address the challenge of the barren plateau. Following this review, we plan to further evaluate and refine our algorithm in the subsequent phase of our research.

Reviewer comment (Presentation):
Poor arrangement with lots of important information in the appendix, leading to difficulty in understanding the paper.

Author response:
**To Point Four:**
I apologize if the current arrangement of our paper has hindered your understanding. Due to page constraints, we placed essential numerical results in the appendix. *If you have any suggestions for improving this aspect, I would greatly appreciate your input and will promptly revise the paper to enhance its structure*.

Reviewer comment (Theory):
The authors do not present any new analytical studies to support their conclusions, and the work presented is entirely numerical.

Author response:
**Analytical results to support our conclusion - the feasibility of the VQLS-based QSVM in the NISQ-era:**

In our study, we employed a variational method that facilitates task-oriented programming and a Hardware-Efficient Ansatz. This approach is specifically designed to accommodate encoding symmetries and bring correlated qubits into closer proximity, thus reducing circuit depth [1]. A key benefit of this strategy is the maintenance of shallow quantum circuit depth, which is critical for mitigating noise— a notable contrast to algorithms developed for fault-tolerant quantum computing [2]. This framework is a crucial factor making our VQLS-based QSVM feasible on NISQ computers.

Another important aspect is the density matrix of the unitaries in the decomposition of matrix $A$, as detailed in Eq. A 4 of [3]. This matrix has a prefactor proportional to $d$ , indicating a $d$ -sparse matrix structure. The precision in estimating the expectation value of each unitary is inversely proportional to $d$. We conducted a comparative analysis of our algorithm's convergence in two scenarios:  : 1) without employing SVD, and 2) with the application of SVD. The empirical results, as illustrated in Figure 10 of our paper, highlight improved trainability of VQLS-based QSVM when SVD is applied before encoding matrix $A$ into the quantum computer. This suggests that the application of SVD effectively reduces the sparsity of the Hamiltonian, thereby enhancing its trainability.

Reviewer comment (Experiments):
It is not known how the approach will scale with the number of qubits required in the setup, as the paper presents only very small numerical examples (3–4 qubits). The method works in that regime, but it is not clear how it will behave for qubit counts required to utilize quantum advantage.

Author response:
**Regarding to the large-scaled numerical results**

VQLS has shown significant scalability on NISQ- device when addressing larger problem sizes with artificial examples, as elaborated in [3]. This discovery underpins our initial step in proving the feasibility of VQLS-base QSVM in the real-world application.

Moving forward, our research will delve into scalability aspects with large-scale qubits, addressing potential challenges and offering analytical insights. We are currently examining literature on variational algorithms' challenges, such as the barren plateau phenomenon, particularly in contexts involving a large number of qubits. 

A thorough investigation, mitigation of challenges, and analysis of the numerical results for large-scale scenarios will necessitate more time. If feasible, I am keen to include these findings in the current paper. I hope for your understanding and thank for your suggestion and support in this matter.

**How it will behave for qubit counts required to utilize quantum advantage**

Quantum computing is able to encode an exponentially increasing amount of data into a linearly growing number of qubits[1]. In our case, we encoded seven examples into three qubits, as mentioned in your comment. However, in the current NISQ era, variational algorithms do not yet demonstrate a clear quantum advantage, as the optimization training process is NP-hard [4]. Should we achieve the development of noise-tolerant quantum computers, it would be possible to directly implement QSVM as proposed in [5]. This advancement could significantly reduce the time complexity from  $\mathcal{O}(\log(\epsilon ^{-1})poly(N, M))$  in classical SVM to  $\mathcal{O}(\log NM)$  [5], marking a substantial leap in computational efficiency.

Reviewer comment (Experiments):
To remedy the scaling uncertainty, the authors could present large-scale numerical examples or attempt to derive analytical results that would give an argument for favorable scaling.

Author response:
**Regarding to the large-scaled numerical results**

VQLS has shown significant scalability on NISQ- device when addressing larger problem sizes with artificial examples, as elaborated in [3]. This discovery underpins our initial step in proving the feasibility of VQLS-base QSVM in the real-world application.

Moving forward, our research will delve into scalability aspects with large-scale qubits, addressing potential challenges and offering analytical insights. We are currently examining literature on variational algorithms' challenges, such as the barren plateau phenomenon, particularly in contexts involving a large number of qubits. 

A thorough investigation, mitigation of challenges, and analysis of the numerical results for large-scale scenarios will necessitate more time. If feasible, I am keen to include these findings in the current paper. I hope for your understanding and thank for your suggestion and support in this matter.

**Analytical results to support our conclusion - the feasibility of the VQLS-based QSVM in the NISQ-era:**

In our study, we employed a variational method that facilitates task-oriented programming and a Hardware-Efficient Ansatz. This approach is specifically designed to accommodate encoding symmetries and bring correlated qubits into closer proximity, thus reducing circuit depth [1]. A key benefit of this strategy is the maintenance of shallow quantum circuit depth, which is critical for mitigating noise— a notable contrast to algorithms developed for fault-tolerant quantum computing [2]. This framework is a crucial factor making our VQLS-based QSVM feasible on NISQ computers.

Another important aspect is the density matrix of the unitaries in the decomposition of matrix $A$, as detailed in Eq. A 4 of [3]. This matrix has a prefactor proportional to $d$ , indicating a $d$ -sparse matrix structure. The precision in estimating the expectation value of each unitary is inversely proportional to $d$. We conducted a comparative analysis of our algorithm's convergence in two scenarios:  : 1) without employing SVD, and 2) with the application of SVD. The empirical results, as illustrated in Figure 10 of our paper, highlight improved trainability of VQLS-based QSVM when SVD is applied before encoding matrix $A$ into the quantum computer. This suggests that the application of SVD effectively reduces the sparsity of the Hamiltonian, thereby enhancing its trainability.

Reviewer comment (Theory):
The authors incorrectly referred to the decomposition in Eq. (2) as an eigen decomposition.

Author response:
**One Question:**
*Regarding Equation 2 in my paper, which details the separating hyperplane, could you please provide more specific information or clarification on the issue you've identified with this equation in the "weaknesses" section?*

Reviewer comment (Evaluation):
Overall, it is hard to be convinced by the results that the work “takes us one step closer to realizing possible practical applications of QSVM on a quantum computer in the NISQ-era.”

Author response:
**Analytical results to support our conclusion - the feasibility of the VQLS-based QSVM in the NISQ-era:**

In our study, we employed a variational method that facilitates task-oriented programming and a Hardware-Efficient Ansatz. This approach is specifically designed to accommodate encoding symmetries and bring correlated qubits into closer proximity, thus reducing circuit depth [1]. A key benefit of this strategy is the maintenance of shallow quantum circuit depth, which is critical for mitigating noise— a notable contrast to algorithms developed for fault-tolerant quantum computing [2]. This framework is a crucial factor making our VQLS-based QSVM feasible on NISQ computers.

Another important aspect is the density matrix of the unitaries in the decomposition of matrix $A$, as detailed in Eq. A 4 of [3]. This matrix has a prefactor proportional to $d$ , indicating a $d$ -sparse matrix structure. The precision in estimating the expectation value of each unitary is inversely proportional to $d$. We conducted a comparative analysis of our algorithm's convergence in two scenarios:  : 1) without employing SVD, and 2) with the application of SVD. The empirical results, as illustrated in Figure 10 of our paper, highlight improved trainability of VQLS-based QSVM when SVD is applied before encoding matrix $A$ into the quantum computer. This suggests that the application of SVD effectively reduces the sparsity of the Hamiltonian, thereby enhancing its trainability.

**Regarding to the large-scaled numerical results**

VQLS has shown significant scalability on NISQ- device when addressing larger problem sizes with artificial examples, as elaborated in [3]. This discovery underpins our initial step in proving the feasibility of VQLS-base QSVM in the real-world application.

Moving forward, our research will delve into scalability aspects with large-scale qubits, addressing potential challenges and offering analytical insights. We are currently examining literature on variational algorithms' challenges, such as the barren plateau phenomenon, particularly in contexts involving a large number of qubits. 

A thorough investigation, mitigation of challenges, and analysis of the numerical results for large-scale scenarios will necessitate more time. If feasible, I am keen to include these findings in the current paper. I hope for your understanding and thank for your suggestion and support in this matter.

**How it will behave for qubit counts required to utilize quantum advantage**

Quantum computing is able to encode an exponentially increasing amount of data into a linearly growing number of qubits[1]. In our case, we encoded seven examples into three qubits, as mentioned in your comment. However, in the current NISQ era, variational algorithms do not yet demonstrate a clear quantum advantage, as the optimization training process is NP-hard [4]. Should we achieve the development of noise-tolerant quantum computers, it would be possible to directly implement QSVM as proposed in [5]. This advancement could significantly reduce the time complexity from  $\mathcal{O}(\log(\epsilon ^{-1})poly(N, M))$  in classical SVM to  $\mathcal{O}(\log NM)$  [5], marking a substantial leap in computational efficiency.

Reviewer comment (Experiments):
The authors could present large-scale numerical examples that would give an argument for favorable scaling under realistic noise models.

Author response:
**Regarding to the large-scaled numerical results**

VQLS has shown significant scalability on NISQ- device when addressing larger problem sizes with artificial examples, as elaborated in [3]. This discovery underpins our initial step in proving the feasibility of VQLS-base QSVM in the real-world application.

Moving forward, our research will delve into scalability aspects with large-scale qubits, addressing potential challenges and offering analytical insights. We are currently examining literature on variational algorithms' challenges, such as the barren plateau phenomenon, particularly in contexts involving a large number of qubits. 

A thorough investigation, mitigation of challenges, and analysis of the numerical results for large-scale scenarios will necessitate more time. If feasible, I am keen to include these findings in the current paper. I hope for your understanding and thank for your suggestion and support in this matter.

Reviewer comment (Theory):
The authors could attempt to derive analytical results to support the feasibility of running QSVM in the NISQ era.

Author response:
**Analytical results to support our conclusion - the feasibility of the VQLS-based QSVM in the NISQ-era:**

In our study, we employed a variational method that facilitates task-oriented programming and a Hardware-Efficient Ansatz. This approach is specifically designed to accommodate encoding symmetries and bring correlated qubits into closer proximity, thus reducing circuit depth [1]. A key benefit of this strategy is the maintenance of shallow quantum circuit depth, which is critical for mitigating noise— a notable contrast to algorithms developed for fault-tolerant quantum computing [2]. This framework is a crucial factor making our VQLS-based QSVM feasible on NISQ computers.

Another important aspect is the density matrix of the unitaries in the decomposition of matrix $A$, as detailed in Eq. A 4 of [3]. This matrix has a prefactor proportional to $d$ , indicating a $d$ -sparse matrix structure. The precision in estimating the expectation value of each unitary is inversely proportional to $d$. We conducted a comparative analysis of our algorithm's convergence in two scenarios:  : 1) without employing SVD, and 2) with the application of SVD. The empirical results, as illustrated in Figure 10 of our paper, highlight improved trainability of VQLS-based QSVM when SVD is applied before encoding matrix $A$ into the quantum computer. This suggests that the application of SVD effectively reduces the sparsity of the Hamiltonian, thereby enhancing its trainability.

Reviewer comment (Experiments):
The motivation for using VQLS to obtain a solution to the hyperplane is to have a robust solution on NISQ devices where noises play the main role and obstruct the original HHL subroutine in the QSVM. However, theoretical analysis and experiments in the paper do not provide enough evidence to support the fact that the proposed method mitigates the effects of noise. The theoretical analysis only considers perfect implementation and the experiments are conducted on an ideal circuit simulator. The suggested robustness of the method might be severely limited in the subroutines requiring precise control of the quantum system, e.g., the Hadamard test. The evidence in the paper does not convince me of the necessity of substituting VQLS for HHL in the QSVM in the first place.

Author response:
**Noise affect the performances of VQLS-enhanced QSVM and HHL-based QSVM**

The fidelity of experimental results in quantum systems, particularly in the context of numerical experiments, has been investigated as detailed in [1]. To the best of our knowledge, the impact of noise on HHL-based QSVM remains unexplored. While there is an experimental implementation of the HHL-based QSVM algorithm presented in [2], the effects of noise in real hardware environments have not been thoroughly examined in [2].

Addressing noise in quantum hardware is a substantial challenge in quantum information theory, with topics like quantum noise correction and mitigation being pivotal areas of research. However, it's important to mention that our current work does not delve into the  effects of noise on quantum algorithms or the strategies for mitigating such noise.

Concurrently, our paper seeks to address a significant research gap: the exploration of VQLS' feasibility in practical, real-world scenarios. This gap was identified after an extensive review of existing literature citing the original research on VQLS in [3]. By focusing on this uncharted area in quantum computing research, we aim to contribute valuable insights and advancements in the field.


Thank you for your constructive feedback. We acknowledge the importance of this topic. The investigation into noise effects in quantum simulations and the development of strategies to mitigate these effects will be a focus in our future work. We also hope that this paper will inspire other researchers to further investigate and contribute to this area.

Reviewer comment (Experiments):
The use of SVD to regularize the data seems not scalable. One of the benefits of QSVM is the ability to deal with high-dimensional data, while the classical SVD regularization can only be performed in small cases. This fact limits the scenarios where SVD might be useful.

Author response:
**How scalable is the SVD regularization?**

The time complexity of SVD is $O(min ( m^{2}n, mn^{2} ))$, as detailed in [4]. The underlying intuition for applying SVD to matrix $A$ is similar to the intuition in the hybrid classical-quantum variational algorithms. In such algorithms, A shallow depth of quantum ansatze is implemented and processed on a quantum computer, effectively reducing the depth of the quantum circuit [5]. However, this benefit comes at the cost of increased time complexity for solution finding in the classical computer.


In the NISQ era, striking a balance between computational resources allocated to quantum and classical computers is pivotal. Our comparative analysis of results with and without SVD application, as presented in Figure 10 in our paper, empirically substantiates the theoretical conclusions drawn in Appendix 1 of [3]. This comparison shows that while SVD regularization increases computational complexity on the classical computers, it offers significant benefits in terms of improving algorithm performance.

Reviewer comment (Experiments):
The lack of consideration of noises also weakens the claims on the effectiveness of SVD. It is unclear whether noises are affecting the performance of SVD.

Author response:
**How do noises affect the performance of SVD?**
The performance of SVD is computed on the classical computer in the data preparation. The noisy data affected the performance of SVD, as detailed in [6],[7].

Reviewer comment (Writing):
Minor typos are present in the paper:
- Page 1, "provided as an *imput* to the quantum hardware"
- Page 7, "IBM-Q aer simulator" => IBM-Q Aer simulator
- Page 7, ", Running at"

Author response:
I am grateful for your insightful comment. The mino types will be corrected in the next version of paper. Please find below my detailed response, reflecting my best effort to comprehensively address your questions: