Paper: Neural Auto-designer for Enhanced Quantum Kernels (ICLR-2024)

Reviewer comment (Novelty):
The problem of quantum kernel auto-designer defined by the authors is only a special case of QAS. QAS has attracted continuous attention with numerous algorithms trying to solve this problem (see a small survey on this problem [1]). The state-of-the-art method [2] on QAS has thoroughly examined their algorithm on both VQE problems and quantum kernels (using MNIST as well). It is unclear why we should focus on only the quantum kernels instead of the general QAS problem.

Author response:
Thanks for your comment. We will primarily address your question from the following two aspects:
1) __Quantum kernel search vs QAS__. Quantum kernel search is not a special case of QAS because we cannot directly apply QAS methods to design the quantum kernel due to the high complexity of evaluating the quantum kernel. Therefore, the one of main contributions of our paper is the introduction of kernel-target alignment (KTA), which greatly reduces the complexity of evaluating the quantum kernel (from $O(n^3)$ to $O(n^2p)$, $n$ is the number of samples, $p$ refers to the dimensionality of the data after dimensionality reduction), making quantum kernel search possible. 
2) __Complementarity__. Quantum kernel search and QAS are complementary as designing a good quantum kernel. On the other hand, quantum kernel search and QAS are all can borrow ideas form Neural Architecture Search (NAS). From this perspective, quantum kernel search cannot be a subset of QAS.

Reviewer comment (Experiments):
The experimental results are weak and lack a proper comparison. Not a single baseline from all the QAS methods is unacceptable since [2,3] and many other QAS papers all take quantum kernels as one of the testbeds for their QAS approaches. The authors constantly mentioned that the work aims to "greatly improve the power of quantum kernels under NISQ setting" but they are not considering noise models or quantum devices.

Author response:
Thanks for your comment. The noise simulation results are displayed in Appendix C.7 of __our original manuscript__. The main results is that our method exhibits good adaptability across different levels of noise and performs well on real quantum devices. It is capable of finding quantum kernel with good performance.

Reviewer comment (Novelty):
Since a randomized quantum kernel has attained outstanding quantum representations and exactly corresponds to a special quantum neural network, the motivation of using an enhanced quantum kernel with automatic neural architecture search is not a key topic.

Author response:
Thanks for your comment. As mentioned in the first paragraph on the page 2 of our paper, “quantum kernels associated with inappropriate quantum feature maps tend to be inferior to that of classical kernels [1]. Even worse, when the quantum feature map involves deep circuit depth, a large number of qubits, and too many noisy gates, the corresponding quantum kernel may encounter the vanishing similarity [2] and the degraded generalization ability [3], precluding any potential computational advantages.” These indicates that designing a good quantum feature map is very important. However, it is not clear how to design a good quantum kernel as there is no prior knowledge to guide us in designing the kernel.

On the one hand, quantum kernel design is quite different from the field of deep learning, where there are many well-performing structures that can serve as basic modules to construct neural networks, such as CNN, RNN, and Transformer. In contrast, quantum machine learning is still in its early stages, and we are uncertain about which well-structured modules can be used to build quantum kernels. This situation is even more challenging in the near-term quantum devices, as it requires considering noise, topology, and various limitations. Designing a good quantum kernel becomes more challenging. In such cases, using automated search methods to design quantum kernels becomes one of the approaches to address this problem. Consequently, many researchers have conducted extensive studies in this area. In the NISQ era, quantum kernels have the potential to be a promising approach for achieving quantum advantage [4]. And quantum kernels also have a solid theoretical foundation [5-7]. Another hot topic is Quantum Architecture Search (QAS) [8-10]. There are also exists some papers related to quantum feature map design suggest that adjusting quantum kernels can improve their performance on downstream tasks [11-13]. 

Therefore, designing better quantum kernels is both urgent and crucial, as it represents a key step towards achieving potential quantum advantage on near-term quantum devices.

To address your concern, we have added a discussion on motivation of our work in the Appendix of the revised version of manuscript.

Reviewer comment (Experiments):
The computation cost of the two-stage optimization for the objective as Eq. (4) is such high that only a small subset of MNIST samples is tested for the experiments. However, the randomized quantum kernel can be simply incorporated into a quantum kernel learning architecture, which can be extended to large datasets.

Author response:
Thanks for your comment. We respectfully disagree with your viewpoint that the computation cost of the two-stage optimization for the objective as Eq. (4) is such high.

The one of objectives of this paper is to significantly reduce the optimization complexity. Therefore, we leverage kernel-target alignment (KTA) and the neural predictor to rapidly evaluate the performance of quantum kernels. Our method has significantly reduced the time cost compared to brute force search (from $O(n^3)$ to $O(n2p)$, $n$ is the number of samples, $p$ refers to the dimensionality of the data after dimensionality reduction). More specifically, in the data collection stage, calculating the KTA is more time efficiency than computing the training accuracy on the Tailored MNIST, Tailored CC and Synthetic dataset (e.g., 288s vs 1138s, 86s vs 331s, and 48 vs 188s). While in search stage, our method is more efficient, as it can complete performance evaluation of 50,000 kernels within 75 seconds. From this point of view, the computation cost of our method would not be very high.

The reason for using a small subset of MNIST samples can be attributed to two main factors. Firstly, it is widely acknowledged in the quantum machine learning to use a subset of MNIST for experimentation [1,14,15]. Secondly, this small dataset is sufficient to demonstrate the effectiveness of our method.

Regarding randomized quantum kernel, we have found a reference [16]. If reviewer confirm that this is the correct paper, we can proceed with further discussion, or if reviewer provide the corresponding reference, we can discuss it accordingly.

Reviewer comment (Experiments):
The performance gain in a selected MNIST data subset cannot be convincing enough to demonstrate the effectiveness of the proposed enhanced quantum kernel.

Author response:
Thanks for your comment. To address your concern, we have increased the dimensionality of the Tailored MNIST dataset to 80 and conducted further experiments. The results are presented in the table below. The results demonstrate our method can still find kernels with better performance on higher-dimensional data.
|   __RBFK__ | HEAK |  TEK| QuKerNet|
| -------- | -------- | -------- | -------- |
| 87.60±0.03 |   73.33±0.00| 	34.68±17.84	| **90.27**±0.53

Reviewer comment (Experiments):
For the empirical simulation, many hyper-parameter settings are configured, which can be particularly fitted to certain subsets but difficult to generalize to all data.

Author response:
Thanks for your comment. We follow the standard setting in Neural Architecture Search (NAS), as the key insight of our work is originates from NAS. On the other hand, no free lunch theory [18] also indicates that for different datasets we need different hyperparameters to get the appropriate performance. Secondly, these hyperparameters are important and commonly used in QAS [8] or NAS [17], which assist in balancing search time and performance gains. Lastly, although there are many hyperparameters in our experiments, most of them are not very sensitive. There are three hyperparameters that are worth noting, namely, $p$, $L_0$ and $N$. Among them, $p$ is the dimension after the dimension reduction of the sample. To some extent, the amount of information that the sample carrying is determined by $p$. $L_0$ and $N$ represent the number of layers and qubits, respectively. $L_0$ determines the expression ability of quantum kernel to a certain extent, and $L_0$ together with $N$ define the size of the search space.

Reviewer comment (Novelty):
Why does the quantum neural architecture search a key topic?

Author response:
Tanks for your comment. Please refer to the response provided for Question 1.

Reviewer comment (Experiments):
Since the two-stage alternative optimization has to seek a locally optimal quantum architecture, the computational cost is significantly high. So, what is the trade-off between the performance gain and optimization overhead? Is it necessary to seek an optimal quantum neural architecture?

Author response:
Thanks for your comment. We respectfully disagree with your viewpoint that the computational cost is significantly high. And the trade-off between the performance gain and optimization overhead originated from finding quantum feature map with good performance within a large exponential search space.

One of the contributions of our paper lies in the introduction of KTA, which greatly reduces the computational cost. Please refer to the answer to question 2 regarding the computational cost. Therefore, our computational burden is not excessively high.

About trade-off between the performance gain and optimization overhead, it is necessary to analyze the specific circumstances and make decisions accordingly. When there are numerous quantum feature maps within a search space that exhibit quantum advantage, we can quickly find a good feature map. However, if only a few quantum feature maps within the space demonstrate good performance, it may require more time to discover them. To better address the reviewer's concern, we have conducted relevant experiments to investigate this aspect. The main results is that, in most cases, increasing the number of fine-tuning iterations is equivalent to increase the optimization overhead, can lead to performance improvements. Detailed results are displayed in the table below.

|    | __$\|\Theta\|=0$__ |  $\|\Theta\|=5$|$\|\Theta\|=10$ |$\|\Theta\|=20$|
| -------- | -------- | -------- | -------- |--------  |
| Tailored MNIST $k = 5$ |   84.53±2.44	|86.27±1.72|	86.27±1.72|	86.80±1.36
| Tailored MNIST $k = 10$ | 85.87±1.43|	86.53±1.49|	86.53±1.49|	86.80±1.36
| Tailored CC $k = 5$ | 91.20±1.47|	91.80±1.72|	93.20±0.75|	94.40±0.80
| Tailored CC $k = 10$ |92.20±1.17|	93.00±1.10|	93.40±1.02|	94.40±0.80
|Synthetic dataset $k = 5$|78.67±3.86|	87.00±4.14|	89.00±1.70|	89.00±1.70
|Synthetic dataset $k = 10$|84.00±3.89|	90.00±3.16|	91.33±1.63|	91.33±1.63
 
To address your concern, we have added the relevant results of to the Appendix of the revised version of manuscript.

Reviewer comment (Theory):
The author mentioned that part of the advantage of the quantum kernel comes from its being computationally hard to evaluate classically. However, the quantum kernel proposed in this paper relies on an explicit quantum feature map, meaning explicit quantum gates and quantum circuit encoding. From my view, on NISQ devices with fewer qubits and shallower quantum circuits, such a quantum feature map is not computationally hard to evaluate classically. Therefore, the reviewer believes that the quantum advantage of the proposed quantum feature map in this paper is not readily apparent. Providing a theoretical explanation rather than solely relying on empirical demonstration would make the argument more convincing.

Author response:
Thanks for your comment. We will separately reply your questions. 
1) __Related to quantum feature map of NISQ devices__. We respectfully disagree with your viewpoint that our approach only suitable for NISQ devices. 

Before addressing your detailed concerns, we first explain our __key motivation__. As mentioned in  the first paragraph on the page 2 of our paper, “quantum kernels associated with inappropriate quantum feature maps tend to be inferior to that of classical kernels [4]. Even worse, when the quantum feature map involves deep circuit depth, a large number of qubits, and too many noisy gates, the corresponding quantum kernel may encounter the vanishing similarity [5] and the degraded generalization ability [6], precluding any potential computational advantages.” These indicates that designing a good quantum feature map is very important. On the other hand, several papers have demonstrated the advantage of quantum kernels in certain tasks or datasets. For example, Ref. [7] rigorously proves the superiority of quantum kernels in the discrete logarithm problem, while Ref. [4] demonstrates their advantage in the synthetic data. These papers indirectly confirm the potential advantage of quantum kernels. This is why our work focus on quantum kernel design.

Our approach is not only applicable to NISQ devices but also to fault-tolerant quantum devices. Currently, we are in the NISQ era, where many applications are implemented on NISQ devices. Therefore, we conducted relevant experiments specifically targeting NISQ devices. We also conduct many experiments on fault-tolerant quantum devices (refers to the page 9 of our paper). Therefore, in the revised version's appendix, we will explain that our method is not only applicable to NISQ devices. 

Additionally, in the second sentence of the first paragraph on the page 7 of our paper, we mentioned that “can be modified to accommodate different quantum hardware platforms, enabling flexibility and adaptability.” This indicates our approach is suitable for different quantum devices, encompassing both hardware-adaptive and problem-adaptive solutions. It is not limited to NISQ devices alone. Moreover, on NISQ devices, quantum kernels can exhibit characteristics that are difficult to simulate classically, such as high entanglement or deep circuits. In such cases, the generated quantum kernels may not be easily simulated classically. On the other hand, since our approach is hardware-adaptive, as devices upgrade and the number of qubits increases, we can search for more complex kernels. These kernels are likely to be difficult to simulate classically.

2) __Quantum advantage with theoretical explanation__: Quantum advantage with theoretical explanation is challenging. 

We primarily aim to demonstrate from both practical and theoretical perspectives that achieving provable quantum advantage is challenging.

*Practical perspective*: We hope to transfer the quantum advantage present in synthetic data to real-world data, but this is often challenging. Currently, there is no literature proposing quantum neural network (QNN) or quantum kernels with provable quantum advantage in practical problems. Firstly, quantum advantage is just one aspect that this paper focuses on. Our main emphasis is on improving the performance of quantum kernels on real-world datasets to narrow the gap with kernels on synthetic datasets. Secondly, quantum advantage is influenced by various factors. For instance, Ref. [4] demonstrates that quantum advantage is related to the structure of the data itself. Additionally, Ref. [8] provides two conditions for achieving quantum advantage, namely, quantum kernels encompasses functions that are computationally hard to evaluate classically, and the target concept resides within this class of functions. Therefore, the quantum advantage demonstrated in this paper is not necessarily obvious.

*Theoretical perspective*: Paper [4] stated that *Learning* is only possible if the distribution, model and/or model selection strategy contains a lot of structure, which is not always easy to analyse theoretically. For example, deep learning can learn a distribution from data, but can not construct this distribution through mathematical formulas. This also highlights that many real-world problems cannot be easily expressed in mathematical forms, which hinders the progress of proving quantum advantage theoretically. However, problems that can be well-defined are possible candidates for demonstrating quantum advantage [4,7].

Reviewer comment (Novelty):
In the field of quantum machine learning, optimizing both the circuit structure and the parameters of variational quantum gates for specific tasks has been previously studied. Papers [1] and [2] introduced a completely differentiable quantum architecture search (QAS) framework, avoiding the vast circuit structure search space, which the reviewer considers one of the issues faced by the proposed QuKerNet in this paper.

Author response:
QAS methods are __complementary__ to our approach, and our solution is not an incremental work on QAS.

Thanks for your comment. QAS methods are complementary to our approach, and our solution is not an incremental work on QAS.

We will address your concerns from two aspects. 
1) __Quantum kernel vs QNN__. Quantum kernel has a rich theoretical foundation. For example, Ref. [7] rigorously proves the superiority of quantum kernels in the discrete logarithm problem, while Ref. [4] demonstrates their advantage in the synthetic data. Ref. [10] proved that kernel-based training is guaranteed to find better or equally good quantum models than variational circuit training in supervised machine learning. In contrast, there is limited research in this aspect for QNN. And quantum kernel is not equivalent to the QNN. Although there is a certain mapping relationship between quantum kernel and QNN, it is maybe that QNN requires an exponential number of qubits to achieve the same effect of the quantum kernel [11]. 
2) __QAS for quantum kernel search is not straightforward__. QAS cannot be adapted to quantum kernel search, primarily due to the requirement of calculating Kernel Target Alignment (KTA) during the evaluation process of kernel search. Obtaining KTA needs to be computed based on the kernel matrix derived from the data. As mentioned in Section 3.2 of our paper, QuKerNet is highly flexible, allowing various QAS techniques to be utilized in QuKerNet. Regarding the explanation of the differences between QAS and quantum kernel search, we will provide supplementary information and clarification in the revised version of this paper.

The core idea of the two works mentioned by the reviewer [1] and [2] is consistent with Ref. [12], which is to continuousize the discrete space to accelerate the search process with advanced gradient-based methods. And in Section 3.2 of our paper, we also discuss various variants of our approach. The essence of our work lies in utilizing KTA and Neural Architecture Search (NAS) to efficiently implement quantum kernel design based on a discrete-continuous optimization problem. The reviewer's suggestion of using differentiable methods for quantum kernel search is also feasible in our framework.

Reviewer comment (Novelty):
While transferring some ideas of QAS to quantum kernel methods can be considered one of the contributions of this paper, the reviewer believes the methods of QAS utilized in the paper are somewhat outdated and lack timeliness. For instance, methods based on reinforcement learning [3] and sample-free approaches [1,2] might enable the model to learn better quantum feature maps.

Author response:
Thanks for your common. We respectfully reminder the reviewer transfer some ideas of QAS to quantum kernel methods is not one of the contributions of this paper.

On the one hand, QAS and quantum kernel search are all borrow ideas from NAS. On the other hand, as mentioned in our response to question 2, quantum kernels and QNNs have inherent differences, which is why we cannot directly apply the QAS approach to quantum kernel search. As mentioned earlier, the main contribution of this work is the introduction of KTA. Evaluating the performance of a kernel requires computing the kernel matrix based on the entire training data, unlike QAS which can use a smaller amount of data or fewer epochs to quickly estimate the performance of a Quantum Neural Network (QNN). Therefore, directly transferring QAS ideas to quantum kernel search is not feasible. However, by leveraging KTA, we greatly reduce the evaluation time of kernels from $O(n^3)$ to $O(n^2p)$ ($n$ is the number of samples, $p$ refers to the dimensionality of the data after dimensionality reduction), making quantum kernel search possible. More concretely, calculating the training accuracy on the three datasets (Tailored MNIST, Tailored CC, Synthetic dataset) takes 1138s, 331s, and 188s, respectively. However, the neural predictor enables us to quickly evaluate the performance of a large number of different kernels. For example, evaluating 50,000 kernels only takes 75s.

We respectfully disagree with your viewpoint that the NAS method utilized in this paper is somewhat outdated and lack timeliness. 

Currently, in the field of quantum kernel design, there is no well-established solution. Considering the time required for kernel evaluation, we have employed the neural predictor with KTA to efficiently search kernels. This is our first step in quantum kernel search, and there is ample room for improvement. In future work, we will explore more advanced NAS approaches to design quantum kernels.

The implementation method, as discussed in Section 3.2, is flexible and open to various approaches. At the same, as mentioned in our response to question 2, QAS methods are __complementary__ to our approach. Therefore, the methods mentioned by the reviewer in [1-3] are all applicable and can be considered for future work to further enhance the performance of the quantum kernel. We also discuss these approaches in revised version of this paper.

Reviewer comment (Experiments):
I did not see a clear benefit of the neural predictor. For each dataset one needs to train a new predictor, which requires M circuit layouts with their labels. Then M' layouts are sampled randomly and scored by the neural predictor. I assume that $M'\gg M$. It would be good to see whether sampling M layouts and scoring them with the actual labels would be worse than using the neural predictor. Also, I would expect that an active learning approach like Bayesian optimisation for the architecture search could be more beneficial since it guides the sampling rather than sampling randomly which can be very suboptimal in this large search space.

Author response:
Thanks for your comment. Before addressing your detailed concerns, we first highlight our key motivation. 1) __Quantum kernel has a rich theoretical foundation__.  For example, Ref. [2] rigorously proves the superiority of quantum kernels in the discrete logarithm problem, while Ref. [3] demonstrates their advantage in the synthetic data. Ref. [4] proved that kernel-based training is guaranteed to find better or equally good quantum models than variational circuit training in supervised machine learning. In contrast, there is limited research in this aspect for Quantum Neural Network (QNN). 2) __Quantum kernel is not equivalent to the QNN__. Although there is a certain mapping relationship between quantum kernel and QNN, it is maybe that QNN requires an exponential number of qubits to achieve the same effect of the quantum kernel [5]. 3) __The potential of quantum kernel__. In the field of quantum kernel, there are two main focus. The first is the performance of quantum kernel, examining its capabilities and limitations. The second focus is whether it exhibits quantum advantage, meaning whether it can outperform classical approaches for certain tasks. Therefore, researching quantum kernels is both feasible and important.

When designing quantum kernels, the only similarity with Quantum Architecture Search (QAS) is that borrowing ideas from Neural Architecture Search (NAS), transforming the design of quantum kernel into a discrete-continuous optimization problem. However, as mentioned in the last sentence of the second paragraph on the third page of our paper “The fundamental difference between neural networks and kernels hints the hardness of directly employing QAS to automatically design quantum kernels”. In order to effectively utilize NAS for quantum kernel design, we introduced Kernel Target Alignment (KTA) to reduce the time complexity (from $O(n^3)$ to $O(n^2p)$, $n$ is the number of samples,$p$ refers to the dimensionality of the data after dimensionality reduction) required for evaluating kernel performance.

We next answer your questions one by one.

Essentially, we are aiming to solve a discrete-continuous optimization problem. In NAS, there are many choices, and we chose to use a neural predictor because it is a mature solution in NAS [6] and has also been applied in QAS [7]. Therefore, we consider the neural predictor as our approach to explore. In the neural predictor, our main consideration is the trade-off between runtime cost and performance of quantum kernels.

1) __Benefit of neural predictor__. *Time cost*:  Evaluating the performance of quantum kernels generated by different circuits through random sampling can be extremely time-consuming. Since the time complexity of evaluating the quantum kernel with train accuracy is $O(n^3)$, and $n$ is the number of samples. For example, calculating the training accuracy on the three datasets (Tailored MNIST, Tailored CC, Synthetic dataset) takes 1138s, 331s, and 188s, respectively. Using a neural predictor can greatly reduce the time cost of evaluating kernel performance (e.g., evaluating 50,000 kernels only takes 75s). Using random sampling limits our ability to explore more kernels.

However, the neural predictor enables us to quickly evaluate the performance of a large number of different kernels. For example, evaluating 50,000 kernels only takes 75s. *Performance*: The performance of the kernels corresponding to the randomly sampled and the ones searched through the neural predictor are shown in Fig. 12 (Appendix C.5). The main result is that the performance of kernels found by the neural predictor is significantly higher than randomly searched kernels on Tailored MNIST and Tailored CC datasets (i.e., [75%, 88%] versus [50%, 75%] test accuracy on Tailored MNIST). However, on the Synthetic dataset, the performance is slightly higher than randomly searched kernels.

It is worth noting that, the neural predictor is not the only way to implement the quantum kernel search. Our goal is to automatically design a good quantum kernel, and the neural predictor is just one way to achieve that. To address your concerns better, we have included new discussions in the revised version of the paper.

2)  **Random sampling vs Neural predictor**. The reviewer’s question refers to $M$ or $M'$, so we will answer it in two steps.

*If you are referring to $M$*: the validation accuracy of 200 kernels selected by QuKerNet and random search are shown in Fig. 12 (Appendix C.5). And the main result is that the performance of kernels found by the neural predictor is significantly higher than randomly searched kernels on Tailored MNIST and Tailored CC datasets, and is slightly higher than randomly searched kernels on Synthetic dataset.

*If you are referring to $M'$*: the time for evaluating $M'$ kernels by randomly search is not feasible. For example, estimating 50,000 kernels on the three datasets would require approximately 56,900,000s, 16,550,000s, and 9,400,000s, respectively. 

3) **Activate learning**: As mentioned in the first sentence of the first paragraph on the page 7 of our paper “Our proposed scheme in QuKerNet offers a general framework that can be seamlessly integrated with various advanced methods and techniques.”. This means we provided a paradigm that is highly flexible. And we also mentioned in the second sentence of the second paragraph on the page 2 of our paper “Specifically, ……tuning variational parameters within the quantum feature maps and employing evolutionary or *Bayesian algorithms* to continually adjust the arrangement of quantum gates within the feature map”. So we can utilize methods such as Bayesian optimization (BO) [8] or genetic algorithms [9] to guide kernel search process. These methods are complementary and compatible with our approach. Additionally, we implemented Bayesian optimization for quantum kernel search using Optuna [10] to address reviewer’s concern. We set the number of iterations to 1000, and the experimental results are shown in the table below. The kernel searched by Bayesian optimization did not surpass the random search approach, which may be due to the limited number of iterations.
|    | __Training data__ | Candidate | Fine-tune|
| -------- | -------- | -------- | -------- |
| Tailored MNIST BO |   82.12±0.27|	83.20±1.66|	85.60±1.44| 
| Tailored MNIST Ours |   82.12±0.27|	85.87±1.43|	86.80±1.36| 
| Tailored CC BO|  87.00±0.00|	89.80±1.72|	93.00±1.41| 
| Tailored CC Ours|  87.00±0.00|	92.20±1.17|	94.40±0.80| 
| Synthetic dataset BO| 	80.33±0.67|	76.67±6.15|	89.34±2.26| 
| Synthetic dataset Ours| 	80.33±0.67|	84.00±3.89|	91.33±1.63|

Reviewer comment (Reproducibility):
What is M' (number of sampled layouts) used in the experiments?

Author response:
Thanks for your comment. In our experiments, $M'$ was indeed set to 50,000, which is consistent with $|S|$. This is a typo and we have made the revision in the revised version of the paper.

Reviewer comment (Experiments):
Can you try against better classical baselines to give a sense of how hard these tasks are and quantify the quantum advantage?

Author response:
Thanks for your suggestion. Before addressing your detailed concerns, let us first recall our statement in introduction, the second sentence of the second paragraph on the page 1, “quantum advantages may arise if two conditions are met: the reproducing kernel Hilbert space formed by quantum kernels encompasses functions that are computationally hard to evaluate classically, and the target concept resides within this class of functions.” This implies that quantum kernels with prior knowledge are likely to exhibit quantum advantage. On the other hand, we can quantify the performance gap between classical kernel and quantum kernel on a given dataset using the metric *geometric difference* proposed in reference [3]. Through this metric, we can demonstrate that quantum kernels have better performance on certain datasets that are well-suited for quantum models. That is why we chose synthetic dataset for our experiments. Additionally, we also hope to achieve quantum advantage on real-world data, which is why we selected Tailored MNIST and Tailored CC datasets.

As for the choice of the Radial Basis Function kernel (RBFK) as a baseline, it is a commonly used baseline in machine learning, which makes it a suitable choice for comparison with our proposed approach. 

To better address the concerns of the reviewer, we have also chosen the more advanced classical kernel, the Neural Tangent Kernel (NTK) [11], as an additional baseline. The experimental results of NTK with maximum depths of $d \in [1,2,3,4,5]$ are presented in the table below. It can be observed that NTK performs better than RBFK on the Tailored MNIST and Synthetic datasets (e.g., 88.54% vs 83.60%, 78.34% vs 71.67%). When compared to our approach, NTK has a slightly better performance on Tailored MNIST (e.g., 88.54% vs 86.80%). However, NTK's performance is lower than our approach on the other two datasets, which to some extent indicates that an adaptive kernel design approach for varying data is superior to a fixed kernel approach. We also found that there is a huge performance gap between NTK and QukerNet on Synthetic dataset (e.g., 78.34% vs 91.33%). This is because the Synthetic dataset is better suited for quantum models rather than classical models.

|    | __RBFK__ | NTK| QuKerNet|
| -------- | -------- | -------- | -------- |
| Tailored MNIST |   83.60±2.52| 	**88.54**±0.27	| 86.80±1.36| 
| Tailored CC |  92.80±0.40| 	49.96±0.00| 	**94.40**±0.80| 
| Synthetic dataset| 	71.67±2.79| 78.34±6.67	| 	**91.33**±1.63|