Neural Auto-designer for Enhanced Quantum Kernels
Authors: Cong Lei
Yuxuan Du
Peng Mi
Jun Yu
Tongliang Liu
Venue: ICLR 2024 poster (2024)
Keywords: Quantum machine learning
kernel learning
quantum kernels
feature map
quantum circuit design
Primary area: general machine learning (i.e., none of the above)
TL;DR: We propose a general quantum kernel design scheme to enhance the performance of kernel learning. This approach is also feasible on Noisy Intermediate-Scale Quantum machines.

Abstract: Quantum kernels hold great promise for offering computational advantages over classical learners, with the effectiveness of these kernels closely tied to the design of the feature map. However, the challenge of designing effective quantum feature maps for real-world datasets, particularly in the absence of sufficient prior information, remains a significant obstacle. In this study, we present a data-driven approach that automates the design of problem-specific quantum feature maps. Our approach leverages feature-selection techniques to handle high-dimensional data on near-term quantum machines with limited qubits, and incorporates a deep neural predictor to efficiently evaluate the performance of various candidate quantum kernels. Through extensive numerical simulations on different datasets, we demonstrate the superiority of our proposal over prior methods, especially for the capability of eliminating the kernel concentration issue and identifying the feature map with prediction  advantages. Our work not only unlocks the potential of quantum kernels for enhancing real-world tasks, but also highlights the substantial role of deep learning in advancing quantum machine learning.

Published as a conference paper at ICLR 2024
NEURAL AUTO-DESIGNER FOR ENHANCED QUANTUM KERNELS
Cong Lei!
Yuxuan Du
Peng Mii
Jun Yu?
Tongliang Liu
'Sydncy AI Centre, School of Computer Science, The University of Sydney *Department of Automation, University of Science and Technology of China
2ID Explore Academy
123 i: corresponding authors (duyuxuan122) 23 233mmmil.comm tongliang liu @ sydney.edu.au)
ABSTRACT
Quantum kernels hold great promise for offering computational advantages over classical learners, with the effectiveness these kemels closely tied to the design of the quantum feature map. However, the challenge of designing effective quantum feature maps for real-world datasets, particularly in the absence of sufficient prior information, remains significant obstacle. In this study, present data-driven quantum feature maps.
approach that automates the design of problem-specins Our approach leverages seleuren techniques to handle high-dimensional data on near-term quantum machines with limited qubits, and incorporates a deep neural predictor efficiently evaluate the performance of various candidate quantum kernels. Through extensive numerical simulations on different datasets, we demonstrate the superiority of our proposal over prior methods, especially for the capability of eliminating the kernel concentration issue and identifying the feature map with prediction advantages. Our work not only unlocks the potential quantum kernels for enhancing real-world tasks but also highlights the substantial role of deep learning in advancing quantum machine learning.
INTRODUCTION
Quantum computing presents a compelling prospect for revolutionizing machine learning, harnessing the unique attributes of quantum mechanics such as superposition and entanglement (Feynman, 2018; Biamonte et al., 2017). While many studies have showcased the ability of quantum computers to reach quadratic or even exponential runtime improvements for solving linear equations and matrix factorizations (Harrow et al., 2009; Rebentrost al., 2014; Lloyd al., 2014; Harrow & Montanaro, 2017; Du et al., 2018; Gilyén et al., 2019; Du et al., 2022a), their practical realization for real- world applications poses significant challenges primarily due to the demanding error-corrected qubit requirements (Babbush et al., 2021; Gouzien & Sangouard, 2021). Presently, quantum hardware has quantum (NISQ) era (Preskill, 2018). characterized by limited
entered the noisy intermediate-scane qubit counts, shallow circuit depth, inherent system noise, and constrained topologies. Consequently, great efforts have been directed towards exploring quantum machine learning (QML) algorithms that can effectively operate on NISQ machines (Cerezo et al., 2021; Bharti al., 2022). Among these efforts, quantum neural networks (Mitarai et al., 2018; Abbas et al., 2021; Du et al., 2021; 2022c; Tian et al., 2023; Qian al., 2022) and quantum kernels (Havlfček et al., 2019; Schuld & Killoran, 2019; Blank et al., 2020) stand out as leading proposals. Celebrated by the flexibility of quantum kernels, proof-of- of-principle experiments have been conducted exhibit their feasibility in solving tasks in various domains (Wu et al., 2021; Haug et al., 2021; Park et al., 2020; Krunic et al., 2022).
The experimental advance spurs the exploration of computational advantages of quantum kernels when executed on NISQ machines. In pursuing this inquiry, recent theoretical study pointed out that quantum advantages may arise if two conditions are met (Kübler et al., 202 the reproducing kernel Hilbert space formed by quantum kemnels encompasses functions that are computationally hard to evaluate classically, and the target concept resides within this class of functions. According to the construction rule of quantum kernels, this means that their power tightly hinges on the quantum feature map. typically composed of input-dependent gates to encode classical data into the quantum state feature space and trainable quantum gates with specified layout. This statement has been

---

Published as a conference paper at ICLR 2024
verified when quantum kemelss are applied tackle well-structured tasks, encompassing classification of synthetic datasets (Huang et al., 2021), discrete logarithmic problems (Liu et al., 2021). quantum phase recognition (Wu et al., 2023b), and specific decision problems (Jäger & Krems, 2023).
Different from well-structured problems, identifying quantum feature maps that satisfy both two conditions concurrently poses a significant challenge for most realistic datasets. As result, quantum kernels associated with inappropriate quantum feature maps tend to be inferior to that of classical kernels (Huang et al., 2021). Even worse, when the quantum feature map involves deep circuit depth, a large number of qubits, and too many noisy gates, the corresponding quantum kernel may encounter the vanishing similarity (Thanasilp et al., 2023a) and the degraded generalization ability (Wang et al., 2021), precluding any potential computational advantages.
To address this issue, several initial approaches have been explored, including the design of problem- specific quantum feature maps (Hubregtsen et al., 2022), rescaling the bandwidth of input data (Shaydulin & Wild, 2022; Canatar et al., 2023), and mitigating the negative impact of system noise and shot error (Wang et al., 2021; Shastry et al., 2022). These approaches are complementary to one another and aim to enhance the performance of quantum kernels. Specifically, in the realm of designing problem-specifie quantum feature maps, two primary streams of research have emerged: tuning variational parameters within the quantum feature maps (Hubregtsen et al., 2022; Glick, 2022; Gentinetta et al., 2023) and employing evolutionary Bayesian algorithms to continually adjust the arrangement of quantum gates within the feature map (Altares- et al., 2021; I; Pellow-Jarman al., 2023; Torabian & Krems, 2023). However, both methods encounter severe caveats. The former overlooks the influence of quantum gate layout (see Table I for supporting evidence), while the latter requests high computational overhead solely by adjusting the layout. Besides, none of them addresses the vanishing similarity issue encountered when processing high-dimensional data and struggles to adapt to the limited topology of NISQ machines. Given these considerations, a crucial question problem-specific feature maps capable of (i) adapting to high-dimensional data into modern quantum machines without encountering the vanishing similarity issue; (it) accommodating both layout and parameter optimization with satisfactory performance?
arises: How can we design a generic method to efficiently construct problem- specific quantum
Table 1: Evidence for the effect of gate layout in performance of quantum kernels. Test accuracies of 1000 quantum kernels on tailored MNIST dataset whose feature map consists of the different gate layouts.
max acc: 82.67%
min acc: 44.67 % avg acc: 68.70 %
std: 8.23%
In this work, we utilize advanced deep-learning techniques to overcome the limitations aforemen- tioned. Specifically, we frame the quantum kernel design discrete-continuon joint optimization problem and then propose a data-driven method dubbed quantum kernel design by neural networks (QuKerNet) to solve this optimization problem, which amounts to effectively and automatically designing problem-specific quantum feature maps. To our best knowledge, this is the first framework that enables the automatic design of kernels by simultaneously considering both circuit layouts and variational parameters. Besides, QuKerNet is more sesource--riendly to modern quantum hardware as it takes into account the topology of qubits and the available number of qubits. Despite that neural architecture search and quantum architecture search also orient to the sescrete-contiuoou joint optimization, adapting these proposals to design quantum kernels not straightforward, because of the expensive computational costs of quantum kernels compared to (quantum) neural networks in collecting accurate predictions of training data. In this regard, we devise a surrogate loss to efficiently optimize QuKerNet.
Two core components of QuKerNet are the incorporation of the feature-selection technique enables QuKerNet to handle high-dimensional data and overcome the limitations posed by near-term quantum machines with limited qubits, thereby ensuring the practical utility and suppressing the vanishing similarity issue. Moreover, the exploitation of neural predictor allows QuKerNet to distill knowledge from different quantum kernels on a given dataset, guaranteeing efficient and accurate performance prediction for the novel quantum feature map. Extensive numerical simulations are conducted to validate the efficacy of QuKerNet.
technique and neural predictor. The
The key motivation of our work. 1) Quantum kernel has rich theoretical foundation. 2) Quantum kernel is not equivalent to the Quantum Neural Network (QNN). 3) Quantum Architecture Search (QAS) for quantum kernel search is not straightforward.
2

---

Published as a conference paper at ICLR 2024
On the one hand, quantum kemne design is quite different from the field of deep learning, where there are many well-performing structures that can serve as basic modules to construct neural networks, such as CNN, RNN, and Transformer. In contrast, quantum machine learning is still in its early stages, and we are uncertain about which well-structured modules can be used to build quantum kernels. This situation is even more challenging in the near-ferm quantum devices, as it requires considering noise, topology, and various limitations. Designing a good quantum kereel becomes more challenging.
Therefore, designing better quantum kernels is both urgent and crucial, as it represents a key step towards uchieving potential quantum advantage on near-term quantum devices.
PRELIMINARIES
2.1 QUANTUM COMPUTATION
Quantum mechanics works in the Hilbert space H with H. & C. where C represents the complex Euclidean space. We use Dirac notation to denote quantum states. A pure quanium state is defined by a vector I-> (named 'ket') with the unit length. The mathematical expression of a state is (a) = = a,e, > a (t) E Cd with E, |a.12 1, where the computational basis (i) stands for the unit basis vector e, E Cd. The inner product of two quantum states (a) and b) is denoted by (a b) or (a, b), where (al refers to the conjugate transpose of (a). A state (a) is in superposition if the number of nonzero entries in is larger than one. Analogous to the 'ket' notation, density operators can be used to describe more general quantum states. Given mixture of m quantum pure states (eba) E Cd with the probability pt and Σxm Pr 1, the density operator p presents the mixed state (p&, (4)) M= as p E Pip: with P; |xbx) (val E Caxd and Tr(p) 1.
The basic element in quantum computation is the quantum bit (qubit). A qubit is a two-dimensional quantum state that can be written as |a) a, |0) + a2 [1). Let (b) be another qubit. The quantum state represented by these two qubits is formulated by the tensor product, i.c., (a) @ |b) as a 4- dimensional vector. Following conventions, (a) @ (6) can also be written as |a, b) or (a) [b). For
clearness, we sometimes denote a) |b) as |a),a b) B. which means that the qubits |a)_ A ((6) g)) is assigned in the quantum register A (B). There are two typical quantum operations. The first one is quantum (logic) gales operating on a small number qubits. Any quantum gate corresponds to a unitary transformation and can be stated in the circuit model, e.g., an N-qubit quantum gate U E (U() satisfies U U = 12x. The second one is the quantum measurement, aiming to extract quantum information such as the computation result into the classical form. Given a density operator p. the outcome m will be measured with the probability pm. Tr(KmpK) and the post-measuremmntt state will be KmpKt PPm with Σ, K|K, =I. II.
Any unitary can be decomposed into a set of basis gates. The basis gate set explored in this study is G= (H, Rxx (a), Ry (3), (z),,, CNOT), containing Hadamard gate, three rotational single-qubit gates along X, Y and Z-axis, respectively, and one two-qubit Control-not gate. The mathematical expression of these five basis gates is H = /2_ 1
(0) exp( (inaa-i0a000 /2) with a E (X,Y 2),
X
1 Y = 1 2 = 2 and 0 E [0, 2m). and CNOT = 10) [0) @ l2 + (
2.2 MECHANISM OF QUANTUM KERNELS
Kernel methods provide a powerful framework to perform nonlinear and nonparametric learning. attributed to their universal property and interpretability. Suppose that both the training and test examples are sampled from the same domain X y. The training dataset is denoted by D (10))))))
X V, where (!()) Rd and (()) E R refer to the i-th example with the feature dimension d and the corresponding label, respectively. A general construction rule of kernel methods is embedding the given input (+(+) E Rd into a high-dimensional feature space, i.e., x()) Rdd R29 with q > d, which allows that different classes of data points can be readily separable. Considering that explicitly manipulating ((x(()) becomes computationally expensive for large q, kernel methods construct a kernel matrix K E Rg* to effectively accomplish the learning tasks in the feature space. Specifically, the elements of K represent the inner product of feature maps with Ky = Kj = (d(x)()), ())( for Vi,j E [m), where such an inner product can be evaluated
) )0
3

---

Published as a conference paper at ICLR 2024
by positive definite function k=x(0, (())) in O(d) runtime. The aim of kernel learning is using the employed kernel K E xR"*" to infer hypothesis h(x())== (0(()((())) with high test accuracy, where w are optimal parameters minimizing the loss function
((w)= = X (w, w) + Elw, ((xl - (0))
The performance of kernel methods heavily depends on the utilized embedding function d(-), or equivalently x(+;)) -). As such, various kernels such as the radial basis function kernel, Gaussian kernel, circular kernel, polynomial kernel, and quantum kernel have been proposed to tackle various tasks.
The mechanism of quantum kernels differs from the classical kernels in designing the feature map d(-). For an N --qubit quantum kernel, the prepared quantum state for the i-th example yields ((((((()) ,))
Ug (x( 0) [0] 00.N where Ug(:, 0) is the specified encoding quantum circuit with trainable parame- ters 0 E θ and being the parameter space. Denote p(x(1), 0) = (((a(()) (1), 0)) (p(x(() 0)). The aim
of quantum kernel learning is seeking quantum kernel W(x, E R with @ E i.e.
W Wey(2z az Tr(p((()) ())p(((() 0)), Vi,j E (m),
to infer hypothesis formulated in Eq. (1) subjecting to higher test accuracies than other kernels. In this regard, the power of quantum kernels is dominated by the exploited (g(-, 0). Notably, the implementation of Ug(, 0) is flexible and diverse. For an N-qubit quantum computer with the basis gate set G, the generic form of the encoding circuit is
Ug(ø, a 0) = II Uy(a))V E SU(("), Vj E [4],
(3)
where (a;)) refers to any parameterized gate in G (e.g., Rx Ry and Rz gates) acting on an arbitrary qubit, a; is an element in the set (), and Vj {CNOT, I2] refers to the fixed gate in G acting on arbitrary two qubits, and L represents the total number of quantum gates.
3 QUKERNET: AUTOMATIC NEURAL DESIGNER FOR QUANTUM KERNELS
Let us first revisit the optimization perspective of quantum feature map design to highlight the shorteomings of previous approaches before moving elaborate proposal. As stated in Sec. 2.2, the performance of quantum kernels heavily depends on the exploited quantum feature map ( 0)) in Eq. (2), which has enormous choices attributed the diversity of @ and the gate arrangement of {U,,, Vi)} in Eq. (3), Taking account into such diversity, the loss function of quantum kernel learning in Eq. (1) should be reformulated as
min L(w, S, = A (w, + E(w ()) - - W.SES
(4)
where (x))(. 0)) (n( 10) SN and U amounts that the gate arrangement S E S is used to build the encoding circuit. Note that locating the global optima (S",0") to minimize L(w, S, 0) is difficuut,, caused by the two aspects: (i) the loss landscape with respect is non-convex; (2) the number of feasible gate arrangements |S] exponentially scales with N and (G).
Towards the optimization difficulty, initial attempts have been made to seek sub-optimal solutions of Eq. (4) in an efficient manner. Concretely, Altares-Lóper et al. (2021) discarded the parameter space and only focuses on searching the optimal gate arrangement S S form Ug: Lloyd al. (2020); Hubregtsen et al. (2022) adopted the fixed circuit layout S and focus on optimizing O. Unlike prior studies, our proposal jointly optimizes the gate layout and parameters to approuch the global minima of L in Eq. (4). In this point of view, all the attempts aforementioned are special cases of our method.
Another aspect overlooked in prior literature is the design of effective quantum kernels for high- dimensional datasets, which presents even greater challenges compared to the low-dimensional case. That is, an improper selection gate arrangements can lead to the vanishing similarity issue, also known as kernel concentration (Huang et al., 2021; Thanasilp et al., 2023a) in which the off-diagonal elements of matrix W progressively diminish as the number of qubits N and the total number of quantum gates 7. increase. Consequently, quantum kernels exhibit lower test accuracy compared to
4

---

Published as a conference paper at ICLR 2024
Step I. Build search space 5
Step 2. Train predictor
Step 3. Search top-k kernels
Step 4. Fine-tune kernels
eacoding
encodingg
Updated
layer
U, V
0. (TTA)
Clussical optimiser
training
predict
Test ave (ho
90
88
CNOT
SVAP
0.5
0.3
Iterations
The workflow of QuKerNet. In step 1, QuKerNet sets up the search space S via the accessible basic quantum gate set G. For example, the set of single-qubit gates includes Rx, Ry, Rz represented by colored hexagons, and the set of two-qubit gates contains CNOT, CZ, SWAP represented by the colored rectangles. In step 2. a pentagram represents a feasible candidate circuit in the search space S. In the training process, M candidate circuits (highlighted by the red pentagrams) are collected and transformed to an image via the proposed encoding method. Meanwhile, KTA of these sampled candidate circuits is calculated according to Eq. (5). These training data are used train an MI.P-based neural predictor. step 3, the optimized neural predictor is employed predict the performance
Figure I:
a large number candidate eircuits sampled from the search space S. Afterward, QuKerNet ranks the predicted KTA and selects the top k candidate circuits. In step 4. for each candidate circuit, QuKerNet randomly replaces parameterized gates. which are used to encode data features, by the variational parameters O, highlighted by the hexagon with shadow. These parameters optimized to maximize KTA. Then we compare their classification accuracy and the circuit with the highest accuracy is selected as the output circuit.
other kernels. Additionally, modern quantum machines with limited and noisy quantum resources make challenging to directly embed high-dimensional data features into the encoding circuit. This is because when extremely large value of L is used to encode the data, executing such quantum kernels on NISQ machines introduces a significant amount of errors, resulting in degraded performance (Wang et al., 2021).
3.1 IMPLEMENTATION OF QUKERNET
Here devise data-driven approach to automatically design the near-optimal quantum kernel for the specified learning task by minimizing L in Eq. (4). Our approach, dubbed quantum kernel design by neural networks (QuKerNet), consists of four steps: search space setup, neural-predictor training, top-k quantum kernels search, and fine-tune. An intuition is depicted in Fig. 1. Conceptually, QuKerNet completes the optimization of Eq. (4) via two-stage learning strategy. In the first stage, QuKerNet estimates the optimal circuit layout S* E S, accomplished by the first three steps. In the second stage, QuKerNet estimates the optimal 0 θ under the searched circuit layout, as completed by the last step. Through decoupling the discrete optimization from the continuous optimization, QuKerNet enables an efficient and scalable way to automatically design enhanced quantum kernels. In the remainder of this subsection, we introduce the implementation of each step and defer the omitted details to Appendix B. Search space setup. This step aims to design suitable search space S in Eq. (4), determined by N and L. Concretely, for a large N and L, most of the candidate circuit layouts S yield a poor performance, due the vanishing similarity issue. To this end, QuKerNet applies the feature selection (FS) method, i.c., Max- Relevance and Redundednn (mRMR) (Peng et al., 2005), to the dataset in the preprocessing step, where dimension-reduucioon function Rdd TRP operates with each example (6) to extract entries from d entries and form the new example (+)) with p < d. This approach not only mitigates the vanishing similarity issue but also facilitates the manipulation of high-dimensional data on NISQ machines. Once the new dataset D = (x(), (1 is prepared, QuKerNet establishes S with the constraint N O(p).
5

---

Published as a conference paper at ICLR 2024
Neural-pun Deurra-prediccor training. This step targets to train neural predictor capture the relation between E and the performance of the corresponding quantum kernel. Specifically, the neural predictor takes a set of circuit layouts in S as input and predicts their training accuracy on the dataset D. The optimization of neural predictor follows the supervised learning paradigm, where the weights of the neural network are optimized to minimize the discrepancy between its predictions and the true training accuracies via gradient descent methods. Through this training process, the neural predictor is expected to provide accurate predictions for the performance of unseen circuit layouts in S.
Nevertheless, unlike QAS, the evaluation of the training accuracy of a quantum kernel with a specified circuit is computationally expensive. Therefore, an alternative strategy is needed to efficiently collect a labeled dataset T for training neural predictors. In our approach, we replace the training accuracy with the kemel-target alignment (KTA) as the label that the neural predictor predicts (Cristianini et al., 2001) since KTA is a reliable surrogate for classification accuracy and can be effectively calculated. Given a circuit layout S and the preprocessed dataset D, its KTA yields
K(D; =
N N E (0) (a, 0;5);
(5)
where Wi, (ab 0; S) refers to the (1, entry of the quantum kernel W in Eq. (2) whose circuit layout is S. g(t) E {0, } is the binary label of the i-th example, and N = nll W || denotes the normalization factor. For multi-class datasets with R classes, its KTA is computed by replacing ((y'() with Ji.j. ie, Jz, 1 if y y(ا); otherwise, Jı. 1 R 1) (Camargo & González, 2009). Refer to Appendix C for the comparison of the runtime cost between KTA and the direct optimization. To facilitate calculation, we set O 0 so that all parameterized quantum gates in S encode the data feature without the trainable parameters O. By calculating KTA of M ~ O(poly(p) different circuit layouts from S in parallel, we obtain the labeled dataset T { K(D; (O,,SS))) )
The neural predictor utilized QuKerNet comprises three essential components: input vectorization, model implementation, and optimization strategy. The input vectorization step addresses the conver- sion of the unstructured format of the circuit layout S into a structured representation that can be processed by deep neural networks. In this regard, we employ the transformation strategy proposed by (Zhang et al., 2021) to accomplish this task. For the implementation of the neural predictor, we adopt the Multilayer Perceptron the underlying architecture (Goodfellow al., 2016). The specific configuration of the neural network, including the number of hidden layers, is determined based on the characteristics of the datasets being considered.
Top-k quantum kernels search. The purpose of this step is to select the most promising quantum kernels in the search space. Firstly, we randomly sample M' circuit layouts 8 from S with S' C and | M'. Then, the trained neural predictor is used predict the K(D; S) with S S'. Afterward, these M' circuits are sorted according to K(D; and the top k circuit layouts are preserved to construct the candidate layout set S..
Fine-tune. This step aims to improve the performance of quantum kernels in S.. To do so, given S E S., we randomly reset m encoding gates in Ux(x) tunable parameters, i.e., (g S) turns to be (n( 0: S) and θ / O. Besides, to ensure the performance of quantum kernels, the parameters are initialized using the average value of corresponding features in the relevant gates. Then fine-tune the quantum kernels to maximize the KTA with the fixed 5 by updating A. After training. we choose the circuit layout with the highest classification accuracy from K fine-tuned candidates as the searched quantum kernel.
Then the quantum kernel W can be obtained based the (8",0") Consequently, the optimal w" can be achieved through w' = Σm 1 ((W" +(W) ),,)), which can be used for kernel learning.
3.2 VARIANTS OF QUKERNET
Our proposed scheme in QuKerNet offers general framework that can be seamlessly integrated with various advanced methods and techniques. For instance, the gate set 9 used to construct the search space can be modified to accommodate different quantum hardware platforms, enabling flexibility and adaptability. Furthermore, different feature selection methods such as mRMR (Peng et al., 2005), Principal Component Analysis (PCA) (Abdi & Williams, 2010) and Locality Preserving Projections

---

Published as a conference paper at ICLR 2024
(LPP) (He & Niyogi, 2003) and other methods (Lei & Zhu, 2018; Yuan et al., 2021; Li et al., 2018; Zhu et al., 2018; 2019) can be employed to handle diverse high- tigh-dimensional datasets effectively. In terms of the neural predictor, alternative backbones and encoding methods can be explored to enhance its performance in capturing the relationship between circuit layouts and kernel performance, i.e., graph neural network (GNN) backbone plus graph encoding strategy (He et al., 2023b), Convolutional Neural Network (CNN) backbone with image encoding (Zhang et al., 2021). Additionally, advanced training methods can be employed to optimize the neural predictor with improved performance, i.e., data augmentation (Jaitly & Hinton, 2013; Huang et al., 2023b), Dropout (Srivastava et al., 2014), early stopping (Caruana al., 2000). Moreover, the fine-tuning process can benefit from the incorporation of more sophisticated heuristic methods, enhancing the overall optimization process.
4 NUMERICAL RESULTS
In this section, we conduct a set of experiments to evaluate the performance of QuKerNet across various datasets. Our primary objectives are to explore the potential merits of the quantum kernel designed by QuKerNet compared to prior quantum kernels and classical kernels, assess the effec- tiveness of QuKerNet in addressing the vanishing similarity issue for large-scale quantum kernels, and evaluate the robustness of QuKerNet in noisy environments. Refer to Appendix B and C for the elaboration of model implementations, datasets, and more simulation results.
Datasets. We benchmark QuKerNet on three datasets, each representing a distinct domain: computer vision, finance, and learning tasks relevant the demonstration of quantum advantages. The first dataset utilized is a tailored version of the MNIST dataset (Lecun & Bottou, 1998), consisting of handwritten digit images. To ensure computational efficiency, we distill 300 images from the original MNIST dataset, focusing on the labels from 0 to 4, and reduce the feature dimension of each example from 784 40 using mRMR. The second dataset employed is tailored Credit Card (CC) dataset (Dal Pozzolo et al., 2017). commonly used for fraud detection and risk assessment in the financial industry. For this dataset, we extracted 200 (100 for each class) samples from the CC dataset and reduce the dimension of each example from 28 to 24 using PCA. Last, we utilize the method proposed in (Huang et al., 2021) to relabel the tailored MNIST dataset (class 0 and 1), creating a synthetic dataset that allows to evaluate the potential advantages of QuKerNet in a controlled manner.
Source code and hardware. All sinulations are conducted using Python, utilizing the PennyLane (Bergholm et al., 2018) 8), PyTorch (Paszke et al., 2019), and the JAX library (Bradbury et al., 201 8). All experiments are run on AMD EPYC 7302 16-Core Processor (3.0GHz) with 188G memory (Ubuntu system). The source code for our implementation will be made publicly available GitHub repository. https: / thuub. c tm1 ab/ /aaaa/2002 2024_0222200222 2024_I022____00 ICLR _uke rNet.
Implementation of QuKerNet. Throughout the entire experiment, utilize the hardware efficient ansatz (HEA) (Kandala et al., 2017) as the backbone to construct the search space of QuKerNet. In particular, as shown in Fig. 2(a). HEA subsumes block-wise layout, and each block is composed of parameterized single-qubit gates and fixed two-qubit gates. In this context, the encoding circuit (K( 0) = 11 U,(a,) = I1 (3), where (3) refers to the j'-th block taking the form as U, = ( R., (B, a (X, Y, 2). Big is an element in the set (æ, 0), and V refers to the entangled layer consisting of CNOT gate and 12 gate. The construction rule of the search space is as follows. For the parameterized single-qubit gates in each block, when the index of qubits i satisfies (nod 2) = we apply the same parameterized gate R2 (each gate in Rg appears at probability of 1/3); otherwise, we adopt another type of parameterized gate Ra.. For the entangled gates each block, when the index of qubit satisfies < N. we pick a two-qubit gate from (CNOT, } uniformly random and apply it to the i-th and (i + 1)-th qubits. This strategy not only facilitates the balance of the size of the search space and the expressivity but also enables an efficient implementation of the circuit on real quantum devices. With slight abuse of notations, in the remained section, we denote Lo as the layer number, contrasting with the original definition of gate number in Eq. (3). For instance, when the Ug encodes all the features of æ. the layer number of UE is seen as and denoted by La = 1 and the relationship between Ly and the number of blocks B, is B Lo [p/N].
Other haper-parameter settings. Here we only introduce the general defer the specific hyper-paramesd settings to the corresponding experiments. Except for the KTA
settings and
7

---

Published as a conference paper at ICLR 2024
Block 1
Block B
layer
10
layer
10 2
R
R
layer
0.8
Test Ace
R2g
RE
Kv
0.7
10
10
Res
Rass
0.6
7
0.5
Rz,
Rix
10
0.4 mRMR
Layer
Layer L
Dimensions
Layers
random
Figure 2: Role feature selection QuKerNet. (a) The depiction hardware efficient ansatz. (HEA), Each layer is composed of Rx gates where the rotational angle on the i-th qubit amounts to the i-th feature of the input data, and CNOT gates acting on the adjacent qubits. (b) Alleviation of vanishing similarity by feature selection. The label of "KV" represents kernel variance. Both "F" and "dimensions' refer to the feature dimension. (c) Comparison with two feature selection methods, i.e. mRMR and random selection.
experiment, N = 8, To E (1,2,3,4,5), M' = 50000,
10, |0| = 20. And the neural predictor optimized by Adam with 0.01 learning rate for 30 epochs, and the criterion used is Smooth L1 Loss. Each setting is repeated with 5 times to collect the statistical results.
Performance metries. We use test accuracy (Acc) and the kernel variance (KV) to evaluate the performance of QuKerNet, which quantifies the generalization ability and inspects the degree of vanishing similarity of the specified kernel, respectively. The mathematical expression of KV for the kernel W is Var(W) E(3 (B(W))?, where the expectation is taken over the input data
In the following, we present our numerical simulation results to exhibit the effectiveness of QuKerNet.
FS is necessary to alleviate the vanishing similarity issue. Here we show how feature selection affects the performance of the tailored MNIST (with 150 training examples sampled from the distilled dataset). The focus the tailored MNIST rather than the rest two datasets is because of its high- dimensionality feature. The results are visualized in Fig. 2(b). The left line chart indicates that when Lo is fixed, p increases, KV continuously decreases. More precisely, when Lo 5, during the process of increasing p from 8 to 40, KV decreases from approximately 0.05 to 0.001. The right line chart reveals that without feature selection, even if Lo 1, the KV is close to 0.0001. By contrast, after feature selection, it is around 0.001 even when Lo 9.
FS versus random pick. To further explore the role of feature selection, we searched for 40- dimensional feature subset from the tailored MNIST using two methods (mRMR and random pick) to construct two new datasets. Additionally, we randomly generated 300 kernels with Lo {1, 4, 5} to conduct kernel lcarning on these two datasets. The box chart in Fig. 2(c) highlights that the data preprocessed by feature selection attains a higher classification accuracy. Taken together, it can be concluded that the feature selection technique can alleviate the vanishing similarity issue to certain extent and improve the learning performance of quantum kernels.
KTA is good surrogate of training accuracy. putational overhead, QuKerNet adopts KTA instead of training accuracy to con- struct the labeled dataset to train the neural predictor, as elucidated in Sec. 3.1. To verify the effectiveness of this replacement, we investigate the correlation between the performance of our proposal under the loss functions in Eq. (4) and Eq. (1) on the same tiny MNIST at two learning stages i.e., QuKerNet- 1 (only optimize the circuit layout S) and QuKerNet-2 (simultaneously optimize the circuit layout S and variational parameters 0), respectively. The tiny MNIST contains 50 train and 50 test examples, with labels ranging from 0 to 4. Each example consists of features selected by mRMR, In this case,
To avoid the expensive com-
PCC
QutcoNees DuKoNNe-11
0.9918
0.9881
0.9766
0.9821
Tikls
KITA
50, SS| 72 and S contains total 3 2 72 different circuit layouts. The restricted search space allows us to explore the whole search space with a thorough analysis. The simulation results are exhibited in Fig. 3. In this case, we use the Pearson correlation coefficient (PCC, higher is better) to evaluate the validity of KTA. From Fig. 3, the PCCs between the training accuracy of kernels found by QuKerNet and the train accuracy of kernels selected by Eq. (1), are 0.9918 (QukerNet-I) 1) and 0.9766 (QukerNet-2). respectively. This indicates we can employ Eq. (4) instead of Eq. (1) to implement kernel learning. Besides, KTA has been demonstrated to be reasonable surrogate as the PCCs between the KTA of kernels selected by QuKerNet and the training accuracy of kernels found through Eq. (1) are both close (i.e., 0.9881 in QukerNet-1 and 0.9821
N N= 4, Lo 1,
Figure 3: Correlation of kernels optimized under different loss functions.
8

---

Published as a conference paper at ICLR 2024
HEAK
TEK
QaKerNes22
100
100
92.8
92_2
100
90.33
$6,
85,8786.8
80
85.4
84
:0
Todl
X1.67
80
68.4
7166
70
Tailore MNIST
Tailonod OC
HEAK
RBFK
TEK
QuKerNer QQukerNes
Figure 4: Numerical results of QuKerNet. (a) The best kernels discovered by TEK. QuKerNet-1, and QuKerNet-2 on tailored MNIST and tailored CC datasets. (b) The training and test accuracy for the best kernels scarched by TEK, QuKerNet-I, 1, and QuKerNet-2 the synthetic dataset. Note that the results of HEAK have no standard deviation as it fixed kernel without any adjustable parameters.
in QukenNet-2). These results validate the feasibility of using KTA instead of training accuracy to construct the labeled dataset to train the neural predictor.
Enhanced performance after each step of QuKerNet. We next exploit the performance of QuKerNet d-
Table 2: The top test accuracy for three datascts on different stages.
Training data
Candidate
Fine-tune
Tailored MNIST
82.13 + 0.27
85.87 + 1.43
86.80 1.36
ifferent learning stages when learn-
Tailored CC
87.00 + 0.00
92.20 1.17
94.40 0.80
ing the three datasets introduced in
Synthetic dataset
80.33 + 0.67
84.00 3.89
91.33 1.63
'Datasets'. We set M = 1000, 500, 1000 for tailored MNIST, tailored CC, and synthetic dataset, respectively. As shown in Table 2, QuKerNet is able to find kernels that perform better than those in the training set, indicating that it has the ability to genuinely grasp certain rules to infer kernel perfor- mance based on the circuit layout. Furthermore, it can be observed that the top performance of the kernels selected by QuKerNet keeps improving in both QuKerNet-1 and QuKerNet-2, demonstrating that optimizing both and 0 is necessary and effective.
Comparison with other kernels. To further verify the performance of kernels found by QuKerNet is superior to unoptimized or individually optimized (S or 0) algorithms,, we compare the performance of five different methods, Radial Basis Function Kernels (RBFK) (Buhmann, 2000), HEA-based quantum kernel (HEAK) (Thanasilp et al., 2023b), Training Embedding Kernels (TEK) (Hubregtsen al., 2022), QuKerNet-1, and QuKerNet-2 in learning three datasets aforementioned (see Appendix A.3 for more details about comparison algorithms). The statistical results of five random experiments conducted on tailored MNIST, tailored CC, and synthetic datasets are shown Fig. Compared to the other algorithms, QuKerNet-2 has achieved the highest test accuracy on three datasets (tailored MNIST: 86.8% tailored CC: 94.4%, synthetic data: 11,33%) showcasing its superiority.
Potential advantages We last apply HEAK, RBFK, TEK, QuKerNet-1, and QukerNet-2 to learn the synthetic dataset, with the purpose of exploring whether the kernels designed by QuKerNet may possess certain quantum advantages. From Fig. 4(b), it can be observed that there is huge performance gap between TEK and the kernel designed by QuKerNet (i.e., 73% versus 91.33%), which demonstrates the importance of optimizing S. Besides, the RBFK achieves very low test accuracy compared to quantum kernels, including QuKerNet (i.e., 71.67% versus 91.33%), implying the potential of QuKerNet compared to classical kernels. Moreover, the small gap between train and test accuracy hints good generalization ability of QuKerNet compared to other kernels. Furthermore, the results of QuKerNet-I and QuKerNet-2 (16% versus 1.67%) suggest that optimizing 0 is crucial to improve the generalization of QuKerNet.
CONCLUSION
We propose QuKerNet to automatically design problem-specific quantum feature maps, which greatly improves the power of quantum kernels under NISQ settings. In contrast with prior advantageous quantum kernels established formation on tasks at hand. This characteristic underpins the potential of QuKerNet to use NISQ machines conquer realistic tasks with computational merits.
well-structured problems, proposal does not require prior in-

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
Jun Yu was supported by the Natural Science Foundation of China (62276242), National Aviation Science Foundation (10220711780011) CAAI-Huawei MindSpore Open Fund (CAAIXSILJJ- Anhui Province Key Research and Development Pro-
2021-016B. CAAIXSJLU-2002-0011 gram (202 (202104000200007), Dreams Foundation of Jianghuai Advance Technology Center (2023- 7M012001), USTC-IAT Application Sci. Tech. Achievement Cultivation Program (JL.05221001 Y), Sci. & Tech. Innovation Special Zone 720-163-14-11-001-0000
Tongliang Liu is partially supported by the following Australian Research Council projects: FT220100318, DP220102121, LP220100527, LP220200949, IC IC190100031. 1. The authors would give special thanks to Xinbiao Wang from Wuhan University for helpful discussions.
REFERENCES
Amira Abbas, David Sutter, Christa Zoufal, Aurélien Lucchi, Alessio Figalli, and Stefan Woerner. The power of quantum neural networks. Nature Computational Science, 1(6):403-409, 2021.
Hervé Abdi and Lynne Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433-453) 2010.
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery data mining, pp. 2633-2633, 2019.
Sergio Altares-lóóez, Angela Ribeiro, and Juan José García-Ripoll Automatic design of quantum feature maps. Quantum Science and Technology, 6(4):045015, 2021.
Ryan Babbush, Jarrod R McClean, Michael Newman, Craig Gidney, Sergio Boixo, and Hartmut Neven. Focus beyond quadratic speedups for error-corrected quantum advantage. PRX Quantum, 2(1):010103, 2021.
Marian Stewart Bartlett, Gwen Littlewort, Ian Fasel, and Javier R Movellan. Real time face detection and facial expression recognition: Development and applications human computer interaction. In 2003 Conference on computer vision and pattern recognition workshop, volume 5, pp. 53-53. IEEE, 2003.
Ville Bergholm, Josh Izaac, Maria Schuld, Christian Gogolin, Shahnawaz Ahmed, Vishnu Ajith, M Sohaib Alam, Guillermo Alonso-Linaje B AkashNarayanan, Ali Asadi, al. Pennylane: Auto- matic differentiation of hybrid quantum-classica computations. arXiv preprint 1rXiv:8811004966 2018.
Kishor Bharti, Alba Cervera-Lierta, Thi Ha Kyaw, Tobias Haug, Sumner Alperin-Lea, Abhinav Anand, Matthias Degroote, Hermanni Heimonen, Jakob S Kottmann, Tim Menke, et al. Noisy intermediate-scale quantum algorithms. Reviews of Modern Physics, 94(1):015004, 2022.
Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, and Seth Lloyd. Quantum machine learning. Nature, 1499(671):195-220 2017.
Carsten Blank, Daniel K Park, June-Koo Kevin Rhee, and Francesco Petruccione. Quantum classifier with tailored quantum kernel. npj Quantum Information, 6(1):41, 2020.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclau- rin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, et al. Jax: Compos- able transformations of python+ numpy programs (vO. 2.5). Software available from http:://github com/google/jax, 2018
Martin Dietrich Buhmann. Radial basis functions. Acta numerica, 9:1-38, 2000.
Jorge E Camargo and Fabio A González. A multi-class kernel alignment method for image collection summarization. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 14th Iberoamerican Conference on Pattern Recognition, CIARP 2009, Guadalajara, Jalisco, Mexico, November 15-18, 2009. Proceedings 14, pp. 545-552, Springer, 2009.
10

---

Published as a conference paper at ICLR 2024
Abdulkadir Canatar, Evan Peters, Cengiz Pehlevan, Stefan M Wild, and Ruslan Shaydulin. Bandwidth enables generalization in quantum kernel models. Transactions on Machine Learning Research, 2023.
Rich Caruana, Steve Lawrence, and C Giles. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. Advances in neural information processing systems, 13, 2000.
Marco Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, et al. Variational quantum algorithms. Nature Reviews Physics, 3(9):625-6)) 244,,,566444 2021.
Olivier Chapelle, Patrick Haffner, and Vladimir N Vapnik. Support vector machines for histogram- based image classification. IEEE transactions Neural Networks, 1065):0055-004,, 1999.
N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. S. Kandola. On kernel-target alignment. In Advances in Neural Information Processing Systems [Neural Information Processing Systems: Natural and Synthetic, NIPS 2001. December 3-8, 2001, Vancouver, British Columbia, Canada], 2001.
Andrea Dal Pozzolo, Giacomo Boracchi, Olivier Caelen, Cesare Alippi, and Gianluca Bontempi. Credit card fraud detection: realistic modeling and novel learning strategy. IEEE transactions on neural networks and learning systems, 29(8):3784-3799 2017.
Yuxuan Du, Tongliang Liu, Yinan Li, Runyao Duan, and Dacheng Tao. Quantum divide-and-conquer anchoring for separable nen-negative matrix factorization. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 2093-2099, 2018.
Yuxuan Du, Min-Hsiu Hsich, Tongliang Liu, Shan You, and Dacheng Tao. Learnability of quantum neural networks. PRX Quantum, 2(4):040337, 2021.
Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Shan You, and Dacheng Tao. Quantum differentially private sparse regression learning. IEEE Transactions on Information Theory, 68(8):5217-2233 2022a.
Yuxuan Du, Tao Huang, Shan You, Min-Hsiu Hsieh, and Dacheng Tao. Quantum circuit architecture search for variational quantum algorithms. npj Quantum Information, 8(1):62, 2022b.
Yuxuan Du, Yibo Yang, Dacheng Tao, and Min-Hsiu Hsieh. Demystify prohlem- power of quantum neural networks multi-class classification. arXiv preprint arXiv:2301.01553 2022c.
Trong Duong, Sang T Truong, Minh Tam, Bao Bach, Ju- Young Ryu, and June-Koo Kevin Rhee. Quantum neural architecture search with quantum circuits metric and bayesian optimization. arXiv preprint arX1v:2206.14115, 2022.
Richard P Feynman. Simulating physics with computers. In Feynman and computation, pp. 133-153. CRC Press, 2018
Ya Gao and Shiliang Sun. An empirical evaluation of linear and nonlinear kernels for text classification using support vector machines. In 2010 seventh international conference fuzzy systems and knowledge discovery, volume 4, pp. 1502-1505. IEEE, 2010.
Gian Gentinetta, David Sutter, Christa Zoufal, Bryce Fuller, and Stefan Woerner. Quantum kernel alignment with stochastic gradient descent. arXiv preprint arXiv:2304.09899) 2023.
András Gilyén, Yuan Su, Guang Hao Low, and Nathan Wiebe. Quantum singular value transformation and beyond: exponential improvements for quantum matrix arithmetics. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pp. 193-204, 2019.
Jennifer Glick. Covariant quantum kernels for data with group structure. Bulletin of the American Physical Society, 67, 2022.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
Élie Gouzien and Nicolas Sangouard. Factoring 2048-bit bit rsa integers in 177 days with 13 436 qubits and multimode memory. Physical review letters, (27(14):14055)) 2021.

---

Published as a conference paper at ICLR 2024
Aram W Harrow and Ashley Montanaro. Quantum computational supremacy. Nature, 549(7671): 203-209, 2017.
Aram W Harrow, Avinatan Hassidim, and Seth Lloyd. Quantum algorithm for linear systems of equations. Physicul review letters, 103(15):15050) 2009.
Tobias Haug, Chris N Self, and Myungshik Kim. Quantum machine learning of large datasets using randomized measurements. Machine Learning: Science and Technology, 2021.
Vojtéch Havlíček, Antonio D Córcoles, Kristan Temme, Aram Harrow, Abhinav Kandala, Jerry M Chow, and Jay M Gambetta. Supervised learning with quantum-enhanced feature spaces, Nature, 567(7747):209-212, 2019.
Xiaofei He and Partha Niyogi. Locality preserving projections. Advances in neural information processing systems, 16, 2003.
Zhimin He, Maijie Deng, Shenggen Zheng, Lvzhou Li, and Haozhen Situ. Gsqas: Graph self- supervised quantum architecture search. arXiv preprint arXiv:2303. 12381,, 2023a.
Zhimin He, Xuefen Zhang, Chuangtao Chen, Zhiming Huang, Yan Zhou, and Haozhen Situ. A gnn-based predictor for quantum architecture search. Quantum Information Processing, 22(2):128, 2023b.
Hsin- Hsin-Yuan Huang, Michael Broughton, Masoud Mohseni, Ryan Babbush, Sergio Boixo, Hartmut N- even, and Jarrod R McClean. Power of data in quantum machine learning. Nature communications, 12(1):2631, 2021.
Yiming Huang, Huiyuan Wang, Yuxuan Du, and Xiao Yuan. Coreset selection can accelerate quantum machine learning models with provable generalization. arXiv preprint arXiv:2309. 10441, 2023a.
Zhuo Huang. Xiaobo Xia, Li Shen, Bo Han, Mingming Gong, Chen Gong, and Tongliang Liu. Harnessing duttof-distriibution examples via augmenting content and style. In ICLR, 2023b.
Thomas Hubregtsen, David Wierichs, Elies Gil-Fuster, Peter-Jan HS Derks, Paul K Faehrmann, and Johannes Jakob Meyer. Training quantum embedding kernels near-term quantum computers. Physical Review A, 106(4):04243, 2022.
Jonas Jäger and Roman V Krems. Universal expressiveness of variational quantum classifiers and quantum kernels for support vector machines. Nature Communications, 14(1):576, 2023.
Navdeep Jaitly and Geoffrey E Hinton. Vocal tract length perturbation (vtlp) improves speech recognition. In Proc. ICML Workshop on Deep Learning for Audio, Speech and Language, volume 117, pp. 21, 2013.
Hao Jiang, Wai-Ki Ching, Ka Fai Cedric Yiu, and Yushan Qiu. Stationary mahalanobis kernel svm for credit risk evaluation. Applied Soft Computing, 71:407-417, 2018.
Abhinay Kandala, Antonio Mezzacapo, Kristan Temme, Maika Takita, Markus Brink, Jerry M Chow, and Jay M Gambetta. Hardware-cfficicen variational quantum eigensolver for small molecules and quantum magnets. nature, 549(7671 2:242-246, 2017.
Zoran Krunic, Frederik F Flöther, George Seegan, Nathan D Farnest-Noble, and Omar Shehab. Quantum kernels for real-world predictions based on electronic health records. IEEE Transactions on Quantum Engineering, 3:1-11, 2022.
Jonas Kübler, Simon Buchholz, and Bernhard Schölkopf. The inductive bias of quantum kernels. Advances in Neural Information Processing Systems, 12111-1220000 2021.
Y. Lecun and L. Bottou. Gradient-bused learning applied to document recognition. Proceedings of the IEEE,
1998.
Cong Lei and Xiaofeng Zhu. Unsupervised feature selection via local structure learning and sparse learning. Multimedia Tools and Applications, 77:29605-29611 2018.
12

---

Published as a conference paper at ICLR 2024
Lorenzo Leone, Salvatore FE Oliviero, Lukasz Cincio, and M Cerezo. On the practical usefulness of the hardware efficient ansatz. arXiv preprint arXiv:2211.01477 2022.
Yangding Li, Cong Lei, Yue Fang, Rongyao Hu, Yonggang Li, and Shichao Zhang. Unsupervised fea- ture selection by combining subspace learning with feature self-representation. Pattern Recognition Letters, 109:35 43, 2018.
Kehuan Linghu, Yang Qian, Ruixia Wang, Meng-Jun Hu, Zhiyuan Li, Xuegang Li, Huikai Xu, Jingn- ing Zhang, Teng Ma, Peng Zhao, et al. Quantum circuit architecture search on superconducting processor. arXiv preprint arXiv:2201 000944, 2022.
Minzhao Liu, Junyu Liu, Rui Liu, Henry Makhanov, Danylo Lykov, Anuj Apte, and Yuri Alexeev. Embedding learning in hybrid quantum-classical neural networks. In 2022 IEEE International Conference on Quantum Computing and Engineering (QCE), pp. 79-86. IEEE, 2022.
Yunchao Liu, Srinivasan Arunachalam, and Kristan Temme. rigorous and robust quantum speed-up in supervised machine learning. Nature Physics, 17(9):1013-017, 2021.
Seth Lloyd. Masoud Mohseni, and Patrick Rebentrost. Quantum principal component analysis.
Nature Physics, 10(9):631-633, 2014.
Seth Lloyd, Maria Schuld, Aroosa Ijaz, Josh Izaac, and Nathan Killoran. Quantum embeddings for machine learning. arXiv preprint wrXiv:2001.03622 2020.
Xudong Lu, Kaisen Pan, Ge Yan, Jiaming Shan, Wenjie Wu, and Junchi Yan. Qas-bench: rethinking quantum architecture search and benchmark. In International Conference on Machine Learning, pp. 22880-22898. PMLR, 2023.
Anil Kumar Mandle, Pranita Jain, and Shailendra Kumar Shrivastava. Protein structure prediction using support vector machine. International Journal on Soft Computing, 3(1):67, 2012.
Kosuke Mitarai, Makoto Negoro, Masahiro Kitagawa, and Keisuke Fujii. Quantum circuit learning. Physical Review A, 98(3):032309, 2018.
Kouhei Nakaji and Naoki Yamamoto. Expressibility of the alternating layered ansatz for quantum computation. Quantum, 5:434, 2021.
Mateusz Ostaszewski, Lea M Trenkwalder, Wojciech Masarezyk, Eleanor Scerri, and Vedran Dunjko. Reinforcement learning for optimization of variational quantum eircuit architectures. Advances in Neural Information Processing Systems, 34:18182-18194, 2021.
Jae-eun Park, Brian Quanz, Stephen Wood, Heather Higgins, and Ray Harishankar. Practical application improvement to quantum svm: theory to practice. In Annual Conference on Neural Information Processing Systems, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Rowan Pellow-Jarman, Anban Pillay. Ilya Sinayskiy, and Francesco Petruccione. Hybrid genetic optimisation for quantum feature map design. arXiv preprint arXiv:2302.02988) 2023.
H. Peng, F. Long, and C. Ding. Feature selection based on mutual information criteria of max- dependency, max-relevance, and min-redundancy. IEEE Transactions Pattern Analysis & Machine Intelligence,
2005.
John Preskill. Quantum computing in the nisq era and beyond. Quantum, 2:79, 2018.
Yang Qian, Xinbiao Wang, Yuxuan Du, Xingyao Wu, and Dacheng Tao. The dilemma of quantum neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2022.
Patrick Rebentrost, Masoud Mohseni, and Seth Lloyd. Quantum support vector machine for big data classification. Physical review letters, 113(13): 130503, 2014.
13

---

Published as a conference paper at ICLR 2024
Maria Schuld and Nathan Killoran. Quantum machine learning in feature hilbert spaces. Physical review letters, 122(4):040504, 2019.
Abhay Shastry, Apoorva Patel, Chiranjib Bhattacharyya, et al. Reliable quantum kernel classification using fewer circuit evaluations. arXiv preprint arXiv:2210.0697) 2022.
Ruslan Shaydulin and Stefan M Wild. Importance of kernel bandwidth in quantum machine learning. Physical Review A, 106(4))042077, 2022.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitring. The journal of machine learning research, 15(1):1929-1955) 2014.
Shengyang Sun, Guodong Zhang, Chaoqi Wang, Wenyuan Zeng, Jiaman Li, and Roger Grosse. Differentiable compositional kernel learning for gaussian processes. In International Conference on Machine Learning, pp. 4837. PMLR, 2018.
Leo Sünkel, Darya Martyniuk, Denny Mattern, Johannes Jung, and Adrian Paschke. Ga4qco: Genetic algorithm for quantum circuit optimization. arXiv preprint arX1v:2302.01303, 2023.
Supanut Thanasilp. Samson Wang, Marco Cerezo, and Zoe Holmes. Exponential concentration and untrainability in quantum kernel methods. Bulletin of the American Physical Society, 2023a.
Supanut Thanasilp, Samson Wang, Nhat Anh Nghiem, Patrick Coles, and Marco Cerezo. Subtleties in the trainability of quantum machine learning models. Quantum Machine Intelligence, 5(1):21, 2023b.
Jinkai Tian, Xiaoyu Sun, Yuxuan Du, Shanshan Zhao, Qing Liu, Kaining Zhang, Wei Yi, Wanrong Huang, Chaoyue Wang, Xingyao Wu, et al. Recent advances for quantum neural networks in generative learning. IEEE Transactions on Pattenn Analysis and Machine Intelligence, 2023.
Elham Torabian and Roman V Krems. Compositional optimization of quantum circuits for quantum kernels of support vector machines. Physical Review Research, 100132111 2023.
Ivor W Tsang, James T Kwok, Pak-Ming Cheung. and Nello Cristianini. Core vector machines: Fast svm training on very large data sets. Journal of Machine Learning Research, 6(4), 2005.
Xinbiao Wang, Yuxuan Du, Yong Luo, and Dacheng Tao. Towards understanding the power of quantum kernels in the nisq era. Quantum, 5:531, 2021.
Sau Lan Wu, Shaojun Sun, Wen Guan, Chen Zhou, Jay Chan, Chi Lung Cheng, Tuan Pham, Yan Qian, Alex Zeng Wang, Rui Zhang, et al. Application of quantum machine learning using the quantum kernel algorithm on high energy physics analysis the lhe. Physical Review Research, (3):033221, 2021.
Wenjie Wu, Ge Yan, Xudong Lu, Kaisen Pan, and Junchi Yan. Quantumdarts: differentiable quantum architecture search for variational quantum algorithms. In International Conference on Machine Learning, pp. 37745-37764. PMLR, 2023a.
Yusen Wu, Bujiao Wu, Jingbo Wang, and Xiao Yuan. Quantum phase recognition via quantum kernel methods. Quantum, 7:981, 2023b.
Changan Yuan, Zhi Zhong, Cong Lei, Xiaofeng Zhu, and Rongyao Hu. Adaptive reverse graph learning for robust subspace learning. Information Processing & Management, 58(6)33 2021.
Anqi Zhang and Shengmei Zhao. Evolutionary-based quantum architecture search. arXiv preprint arXiv:2212.00421, 2022.
Shi-Xin Zhang, Chang- Yu Hsieh, Shengyu Zhang, and Hong Yao. Neural predictor based quantum architecture search. Machine Learning: Science and Technology, 2(4):045027, 2021.
Shi-Xin Zhang, Chang Yu Hsich, Shengyu Zhang, and Hong Yao. Differentiable quantum architecture search. Quantum Science and Technology, 7(4):045023, 2022.
14

---

Published as a conference paper at ICLR 2024
Xiaofeng Zhu, Cong Lei, Hao Yu, Yonggang Li, Jiangzhang Gan, and Shichao Zhang. Robust graph dimensionality reduction. In IJCAI. pp. 3257-3263, 2018.
Xiaofeng Zhu, Rongyao Hu, Cong Lei, Kim Han Thung, Wei Zheng, and Can Wang. Low-rank hypergraph feature selection for multi-output regression. World Wide Web, 22:517-531, 2019.
15

---

Published as a conference paper at ICLR 2024
The structure of our Appendix is as follows. Appendix A gives introduction to the related work, noise quantum system, comparison algorithmms we used as well as vanishing similarity of quantum kemnels preliminaries of our paper. Appendix B provides more details of our QuKerNet framework introduced in 3.1 of the main text. Appendix C provides more experimental details and results to help us better understand the capability of QuKerNet.
A PRELIMINARIES
This section serves as an introduction to provide essential background knowledge that will enhance our understanding of this paper. The key areas covered include the related work, the noise quantum system, the comparative algorithms employed in this study, and the phenomenon of vanishing similarity of quantum kernels.
A.1 RELATED WORK
Prior literature most related our work can be classified into two categories: quantum kernels and quantum architecture search (QAS). Here we separately review the connections and differences between these studies and our work.
Kernel-based quantum algorithms. Previous studies of quantum kernels can be divided into two groups. The first group is cata-independent quantum kernels (Blank al., 2020; Huang et al., 2021; Thanasilp et al., 2023b; Huang et al., 2023a), which use fixed circuit to embed the data. These quantum kernels may not have good prediction as the quantum feature map may be inappropriate for the manipulated dataset. Another group is data-dependent quantum kernels (Glick, 2022; Lloyd et al., 2020; Hubregtsen et al., 2022). where the quantum feature is customized to the given dataset. The approaches of designing data-dependent quantum features either focus modifying the circuit layout or tuning variational parameters, indicating that these methods can only achieve local optimal solutions. QuKerNet differs from prior works in the pursuit of the global optima, by simultaneously considering the circuit layout and variational parameters. In this perspective, our work is highly generalized version of prior kernel algorithms, where they are all special cases of our algorithm.
QAS. QAS aims to automatically design the architecture of quantum neural networks that can attain high performance for a specified learning task. Essentially, QAS discrete-continuond joint optimization problem. Depending on the different optimization methods, QAS can be classified into heuristic-based QAS (Zhang & Zhao, 2022; Sünkel et al., 2023), reinforcement learning based QAS (Ostaszewski et al., 2021), Bayesian- QAS (Duong et al., 2022), and other methods (He et al., 2023b; Zhang et al., 2021; He et al., 2023a; Zhang et al., 2022: Lu et al., 2023; Wu et al., 2023a; Du et al., 2022b; Linghu et al., 2022). Different from QAS, QuKerNet orients enhance quantum kernels. The fundamental difference between neural networks and kernels hints the hardness of directly employing QAS to automatically design quantum kernels.
A.2 NOISY QUANTUM SYSTEM
Besides Dirac notation, the density matrix can be used to describe more general qubit states evolved in open quantum systems. Formally, the density matrix of n-qubit pure state (5) is p = (&) (N1 C²" x2", where (wl = | refers to the complex conjugate transpose of (1)). For an ensemble of pure states of quantum system, {pj, (by)) 1 with PA > O, where (w)) is one state of this quantum system and j an index, Σi= Pj = 1, and (ıby) E C²" for j E [m]) its density matrix 1=1 PiPj with Pj (2) (byy and Tr(p) 1.
A.3 MECHANISM OF CLASSICAL AND QUANTUM KERNELS
In the main text, we adopt three kernels, i.e., radial basis function kernel (RBFK), hardware efficient amsatz-based quantum kernel (HEAK), and training embedding quantum kernel (TEK), benchmark the performance of our proposal. For completeness, here we provide the necessary backgrounds of three kernels.
16

---

Published as a conference paper at ICLR 2024
Radial basis function kernel. RBFK is common kernel used in machine learning. The mathemati- cal form of RBFK is
Kij = exp(->/lz - 11)
(6)
where 7 is hhper--paaae parameear RBFK commonly used in various domains such as image classifi- cation (Chapelle et al., 1999), face detection (Bartlett et al., 2003), text classification (Gao & Sun, 2010), protein structure prediction (Mandle et al., 2012), and many other areas (Jiang et al., 2018).
Hardware efficient ansatz-based kernel.
Un (n( 0) (3)
j=1
where (3) refers to the j-th block taking the form as = (c a (X,Y, Big
an element in the set {x, 0), and V refers to the entangled layer consisting of CNOT gate and ll gate. And HEAK is widely used in the fields of quantum machine learning (Nakaji & Yamamoto, 2021), quantum chemistry (Kandala et al., 2017), and combinatorial optimization (Leone et al., 2022)
due to its implementability and expressibility on NISQ devices.
Rx(
Rx (x1)
Rx (21)
Ry (0:)
Ry (0s)
(x()))
Rx (x2)
Rx
Ry (02)
Rx (96)
(x(
Rx (1r3)
(xa)
Ry (0s)
(2x)
Rx(
Rx (2x)
Rx (2a)
Ry (0x)
Rz (07)
(a) HEAK layout
(b) TEK layout
Figure 5: HEAK layout and TEK layout. The æj is corresponding to the j-th feature in sample a(0)) (a) The layout of HEAK. And each dashed region is a block that is used to encode up to N features of ( (b) The layout of TEK. The layout of the dashed region is used for encoding data, while the solid line portion represents variational parameters used to adjust the performance of the kernel. A trainable module is connected after each block of HEAK. The dashed area and the solid line area together form block.
Training embedding kernels. TEK is a method that only optimizes the 0 for given S. In this paper, TEK is designed based on HEAK, and their layouts are shown Fig. Therefore, takes the following mathematical form:
Un( 0) = II Ú,
where (3) is the same to the definition in Eq. (7). While (-) is a flexible layout used to adjust the quantum states of encoding, and 7 represents the variational parameters. In this study, we use the circuit layout proposed in (Hubregtsen et al., 2022) to construct U.). So Uy(7) = (& Ry (rrs)) Yij is an entry of 0, and V refers to the entangled layer. And for the each index of qubit i, we apply the CRz gate to the i-th and (i + 1) (mod N) N)-th qubits. TEK is suitable for image classification (Lloyd et al., 2020) regression tasks (Liu et al., 2022).
In summary, RBFK and HEAK are static kernels with deterministic feature mapping. While HEAK and TEK are quantum kernels and these kernels are built upon the circuit layout S.
A.4 4 VANISHING SIMILARITY OF QUANTUM KERNELS
The vanishing similarity of quantum kernels is that the size of the problem increases, the difference between kernel values becomes increasingly small, which means that kernel values will tend towards the same value. More precisely, vanishing similarity can be formally defined as follows:
82
P W. - E(W,,
5) <
3 0(16bb)
(9)
82
17

---

Published as a conference paper at ICLR 2024
where b 1. If Var[Wa,] O(1/b") and for all Waj. Eq. (9) holds true, identify that this a phenomenon of vanishing similarity. In general, there are four main triggers that lead to vanishing similarity, including high expressibility, global measurements, entanglement and noise (Thanasilp et al., 2023a). When this phenomenon occurs, it becomes challenging to extract meaningful information from the quantum states in reasonable amount of time. This is because distinguishing subtle differences between quantum states to evaluate quantum kernels requires an exponential number of measurements. Therefore, vanishing similarity leads the poor performance of quantum kernels. To address this issue, one should be avoid to design a highly expressible, entangled quantum kernel. For the global measurements-induee quantum kernel to remit it (Liu et al., 2021).
vanishing similarity, one can employ moblem-speciine
B IMPLEMENTATION DETAILS OF QUKERNET
In this section, we elaborate on the missing details of QuKerNet in the main text. In particular, we separately present the implementation details of each stage of QuKerNet, followed by the summarization of its Pseudocode.
Search space setup. We visualize the construction rule of search space in Fig. 6. Specifically, given dataset D (&(+), (f J==1. there are several strategies to load the classical data into an --qubit quantum system. Suppose that the feature dimension of (9) is p. In the case of N = p. the first way element-wise encoding. where the variational quantum gates operating with j-th qubit load the j-th feature (+), i.e., a = a with a is defined Eq. (3). An alternative way is random encoding, where each variational quantum gate loads one non-repetitive feature of (+))
20)
Data(nxd)
Data(uxp)
(00
Feature selection
260)
layer
Figure 6: The step of QuKerNet. First, we preprocess the data through feature selection. Next, we design the search space for the circuits based on relevant rules. Then, we encode the preprocessed data using different circuits sampled from the search space.
We next consider the scenario with < N and design two encoding approaches. First, for the variational quantum gates acting the j-th qubit with j E [N], the encoded data feature is 2) with j (mod p). The second approach involves random parameters. For the variational quantum gate operating with the first p qubits, we set a, = for Vjj [p]. Besides, for the rest N qubits, the parameters the variational quantum gates are randomly and uniformly sampled from the interval [0, 2m).
Last, in the case of p > N. design three encoding methods exhibited Fig. The first way is sequential encoding, which encodes features sequentially. The second way is chain encoding, which utilizes the chain queue to load features. The third way random encoding. In this case, the Oz is set to the value the feature randomly selected from (x),
We note that there are multiple ways of using random encoding to form multi-layer encoding layout. The first solution replicating the entire layout of single block multiple times. Another solution is expanding each dotted area in Fig. 8 multiple times. It is also feasible to repeat the layout of the individual dotted areas in Fig. for the varied times.
Neural- predictr training. encoder and the deep neural networks. The former intends to convert the information circuit layout into format that can be processed by neural networks. The latter is employed to distill the knowledge
adopted in QuKerNet is composed of two parts: image
18

---

Published as a conference paper at ICLR 2024
Ryy
Rxp
Rya
R
Ryz
Rxa
Rxi
Rxp
Ryp
Rrg
Rij0
Ray
Rijo
Riy
Ryg
Reg
Rggy
R
Rys
R
Ryg
Ray
Rys
Riss
Ras
Ris
Rej
Rx
Rig
(a) sequential encoding
(b) chain encoding
(c) random encoding
Figure 7: Various encoding methods. The is corresponding to the j-th feature in sample o (*)
Ray
R
Rat
Rit
Rag
Ri
Rapa
Riza
Rizo
Rxe
Re10
Riio
Roy
Rizg
Riz
Ray
R
Ran
Reg
Riss
Ria
Ras
Rx1
Rev
Figure 8: Repeated encoding scheme. The x, is corresponding the j-th feature in sample a()
between circuit layout (i.e., quantum feature map) and the corresponding performance. In the following, separately explain their implementations.
Rx
Rx
Ry
Channel
Height N
Rx
Ry
Rz
Rx
Rx
Decomposttion
Rz
Rx
Rx
Rx
Channel
Width 2L/N
Figure 9: Representation of quantum circuit layout. The number of qubits N 4, and the total number of rotation gates = First, the input circuit can be divided into five separate layouts, each layout containing only one type of gate. Then, these layouts are individually encoded into images, with each image corresponding to channel. Finally, the images from the five channels are combined to form the final output image.
The implementation of the image encoder is shown in Fig. In particular, we encode circuits into images based on the scheme proposed in (Zhang et al., 2021). For circuits with varied depth, fill the encoded images with blank pixels to the maximum width of images in this set to unify the size of these images (refer Fig. 10(a)). Specifically, N seen the height of the image, and 2L/N corresponds to the width of the image. In addition, each type of gate associates with an image channel. Note that CNOT gate corresponds control qubit. And there are only two pixel values in each channel, that is, O and 1, which indicates whether gate has been applied or not (1 represents gate has been applied, means there is either no gate or la gate has been applied on this position), and the position of the pixel is identical with the position of this gate in quantum circuit.
channels, because includes a target qubit and
The implementation of the neural network adopted in QuKenNet is depicted in Fig. 11. That is, simple Multilayer Perceptron (MLP) is employed as the regression model. This model is a 2-layer MLP model, including one hidden layer and one output layer. And the total number of trainable parameters of this model is 1280L + 257, where L refers to the total number of quantum rotation gates. Adam optimizer is applied in experiments and the learning rate is set 0.01, and the batch
19

---

Published as a conference paper at ICLR 2024
Rx
Ry
load data
Ry
Rg
KTA
Rx
Rx
Rx
Rx
Top circuit
Rx
A
load data
Ry
KTA
Rx
Figure 10: Some details of step and 4 in QuKerNet. (a) The left represents data collection for neural circuits sampled from S will encoded into images a uniform size (with the width determined by the maximum depth among circuits). Then, these circuits also are used to load the data and calculate the KTA using Eq. 5. Finally, the pairs of images and KTA can he used to train the neural network. (b) The right area is the fine-tuning stage, For each of the & circuits found through the QuKerNet- 1, the QuKerNet performs [0] different parameterized gate replacements. In each replacement, the number and positions of replaced gates remain same & circuits.
network training. First,
size 32. Besides, in order to achieve better prediction results, we magnify the value KTA 10 times in each data pair.
input:
[322,58866]
InputLayer
output:
(332,5,8,6)
input:
1(32,5,8,6)
Reshape
output:
[(32,240)]
input:
((32,240)]
Dense
output:
[(32,128)]
input:
[322]288]
ReLU
output:
[(32,128)]
input:
[(32,128)]
Dense
output:
[(32,1)]
Figure 11: MLP based model for regression. [32.5,8,6] corresponding to the batch size, the number of gate types, the number of qubits N, and the depth of the circuit (refer to the width in Fig. 9), respectively.
Fine-tune. Recall that QuKerNet exploits two random methods in this stage to enlarge the parameter space to improve the learning performance. The first random operation refers to the number (m e (1.2..., L/ 1}) of reset parameterized gates in U2(x) that is random in each processing of fine-tune. And the second random operation is that the positions of reset gates are randomly selected. Combined with the above two random operations, QuKerNet can explore larger parameter space. The details of fine-tuning is shown in Fig. 10(b).
For clarification, summarize the pseudocode of QuKerNet.
20

---

(01) and Var (x: (8) is the Published as a conference paper at ICLR 2024
Algorithm 1: Pseudo code of QuKerNet.
Input: a E Rnxd M, M Lo. N, k and [
Output: 5* and :
1 Conducting feature selection on æ to get à E R
Sampling M circuits from S based on Lo. N,, and encoding M circuits to images as well as calculating KTA by Eq. (5);
Train the predictor on training set of M pairs (image, KTA);
4 Sampling M' circuits from S based on Lo. N, and encoding them into images:
Utilizing the predictor to predict the KTA of M' circuits
6 Selecting k circuits with the top predicted KTA from M' circuits. Fine-tuning |+| different O for each the circuits:
7 To validate the test accuracy of kc[@] kernels æ, and pick the kernel with the highest test accuracy.
C MORE SIMULATION RESULTS
In this section, we will introduce some experimental details, including the dataset, hyperparameter settings, and metric. Next, we present additional experimental results to further demonstrate the effectiveness of QuKerNet.
C.1 DATASETS
MNIST is dataset of handwritten digits labeled from 0 to 9, which contains 60,000 grayscale images of 28 28 resolution for training and 10,000 images for testing. While CC financial dataset to identify whether a credit card is fraudulent or not, i.e., binary classification task, with total of 284,80 807 samples (492 frauds), each of which consists of 28 features.
C.2 SYPER-PARAMETEE SETTINGS
Here, we detail the hyper-parameter settings other classical and quantum kernels comparison. RBFK. If there is no special explanation, we set E 2,,4,3,4,5)) 4 Var(z:) variance all the features j 1...., d from all the data points x (1)), ,....,.""" .))))...(.)))
HEAK. In our experiments, a = X. For the entangled gates in each block, when the index of qubit i satisfies N, we apply the CNOT gate the i-th and (i + 1)-th qubits. While the Big corresponds to one feature in (().
We randomly initialize each Yi with value uniformly sampled from the interval [0, 2m). ). and set the epoch to 30 for optimization. Meanwhile, we employ gradient descent optimization with learning rate of 0.2 to minimize the negative KTA of TEK.
TEK.
Both TEK and HEAK employ a sequential encoding strategy (refer to Fig. 7(a)), with Lo = 1 to encode data.
C.3 METRIC
The Pearson correlation coefficient (PCC) is defined as:
PCC = cov(X, YY
(10)
σxoy
where X and Y are given pair of random variables (for example, test accuracy and KTA), the cov the covariance, σx and σy are the standard deviation of X and Y. respectively. The value of PCC ranges from to which implies the degree of correlation between X and Y. The larger the absolute value of PCC, the stronger the correlation (-1 means they are perfectly negatively correlated, while 1 indicates they are perfectly positively correlated).
21

---

Published as a conference paper at ICLR 2024
C.4 THE TIME COMPLEXITY OF CALCULATING KTA VERSUS CALCULATING TRAIN ACCURACY
Here, we analyze the time complexity of obtaining KTA and the training accuracy of a kernel to demonstrate the computational efficiency of calculating KTA.
| ( as stated in (Tsang et al., 2005), the time complexity of training and test processes of the kernelised Support Vector Machines are all (n²)), where the number of the data. So the time complexity of calculating train accuracy is O While the time complexity of computing KTA is O( p). as there are * n entries kernel matrix W and getting each entry needs (p) time complexity, where p refers to the number features of 2(6), In general, p < which indicates that the computational efficiency of KTA is much higher than that of train accuracy.
For given dataset D {-((),
C.5 QUKERNET VERSUS RANDOM SEARCII
Random senrch
Random search
Random search QuKerNet
20
QuKerNet
QuKerNet
Probabllity density
Probability density
Probability density
10-
15
10-
8
6-
2
0.5
0.6
0.7
0.8
0.9
0.6
0.7
0.8
0.9
0.5
0.6
0.7
0.8
0.9
Validation accuracy
Validation accuracy
Validation accuruey
(8) Tailored MN
(b) Tailored CC
(c) Synthotic datasct
Figure 12: Validation accuracy histogram between eircuit layout from random search and QuKerNet.
To further demonstrate the superiority of the QuKerNet over the random search method, we conducted specific validation the three datasets introduced in 'Datasets' evaluate the performance of the kernels based on circuit layouts searched by QuKerNet-1 and random search. In this case, N =, Lo {1,2, 2, 5), |S| 50000, k 200. The performance of 200 kernels selected by QuKerNet-1 and random search are shown in Fig. 12. On tailored MNIST and tailored CC datasets, most of the kernels selected by QuKerNet-1 have better performance than the randomly searched kernels (i.e., [75%. 88%] versus [50%, 75%] on tailored MNIST), For the synthetic dataset, although the performance of the majority of kernels obtained through these two methods overlaps, there are still a few kernels that outperform those found by random search. The above findings are sufficient to demonstrate that our method is indeed capable discovering circuit layouts that are superior to those found through random search for constructing better kernels.
C.6 THE OPTIMAL CIRCUIT LAYOUT FOUND BY QUKERNET ON SYNTHETIC DATASET
Due to space constraints in the main body of this paper, here provide the optimal circuit layouts obtained through QuKerNet on the systhetic dataset introduced 'Datasets..
C.7 NOISE SIMULATION RESULTS
Performance under real device noise. further verify the adaptability of QuKerNet, we compared the performance of the quantum kernel searched by QuKerNet in real device noise and noiseless cases. And we employ ibmgg_quiit to conduct experiments, the noisy parameters are shown in Table 3. Due to the time-consuming nature of noise experiments, in order to obtain experimental results within reasonable time, we took one-third of the synthetic data to generate new synthetic data. Furthermore, PCA is used reduce the dimensionality of the data to 8 dimensions. In this case,
22

---

Published as a conference paper at ICLR 2024
Ry
Rx
Rx
Ry
Ry
Rz
Rx
Rx
Rx
Ry
Ry
Rx
Rx
Ry
Ry
Rz
Rx
Rx
Rx
Ry
Ry
Rx
Rx
Ry
Ry
Rz
Rx
Rx
Rx
Ry
Ry
Rx
Rx
Ry
Ry
Rz
Rx
Rx
Rx
Ry
Figure 13: The layout of selected best encoding circuit for synthetic dataset (the layout of cach dashed section will be repeated five times).
M 500, N 4, To E (1,2,3), 20000, 10, = All experiments were run three times with different random seeds (7 E (1, // Varlæ, (8) )) for RBFK). Experimental results are shown Fig. 14(a). Fig. 14(a) shows that the performance gaps of the kernels between the noise and noiseless cases identified by QuKerNet
not greater than 3.5% (i.c., 61.67% versus 63.33% for QuKerNet-1, 70% versus 73.33% for QuKerNet-2), Although HEAK does not show any difference in performance between the noise and noiseless conditions, its performance is significantly lower than the kernels selected by QuKerNet (i.e., 40% versus 70% in noise situations). These results indicate that QuKerNet has good adaptability to real noise conditions and can be effectively used on practical devices.
Table 3: The noisy parameters of ibmq_quito, respectively. represents qubit frequency. "RE" refers to readout error of given qubit.
quito. Ti and 122 the longitudinal and transverse relaxation time
Parameter
Q1
Q2
Q3
Q4
Tuss)
48.21
60.20
30.10
70.56
Ti (us)
26.82
89.08
14.46
13.10
F(GHz)
5.30
5.08
5.32
5.16
RE
0.0443
0.0220
0.1111
0.0450
Performance under different noise levels.
95
80
Naise
Noiseless
70
63,33
85
61,67
Test Aoo
60
48L33
Text Apc(%)
75
50
48,3368,33
noise level 1
65
40
noise level 3 noise level 4
30
20
45
10
BI.AK
QukarNet-1 Quleerea
stages QuKerNNtt
Figure Noise results of QuKerNet. (a) The best kernels discovered by TEK, QuKerNet-1, QuKerNet-2 on systhetic dataset in real device noise and noiseless situations. Note that the RBFK has the same results in noise and noiseless cases as it is classical kernel method, (b) The effect of different noise levels for QuKerNet small part of the tailored CC dataset. S1. S2 and S3 represent the training stage. candidate stage, and fine-tuning stage, respectivcly. The optimal performance of kernels in each stage shown the line chart.
To further verify the adaptability of QuKerNet, we inspected the performance of the quantum kemel searched by QuKerNet in different noise levels. As noise simulation is mone-consuming, employ the PennyLane to simulate the noisy quantum models, and we only select 40 samples on labels and 1 from the tailored CC with 20/20 train/test split construct the toy CC. Then employ PCA to
23

---

Published as a conference paper at ICLR 2024
reduce the dimension of toy CC to 12 dimensions. And we set up 4 noise levels are shown in Table 4. Besides, we set M = 1300, N = 4, Lo E (1.2.3), |S| = 50000, &o = 10, |9| = 10 to conduct shown in Fig. 14(b). From
experiments. And the statistical results of five random trials Fig. 14(b), we know that although the performance of the kernels in training data will decrease with the increasing noise (i.e., from 79% to 46%). QuKerNet can still find kernels with high performance (i.e., around 90% test accuracy) in all noise levels, indicating that QuKerNet has better robustness and adaptability.
Table 4: The details of four noise levels (DQEP refers to depolarizing quantum error probabilities).
level
lcvel 2
level 3
level 4
DQEP of I-uubit gate DQEP of 2-qubit gate
0.005
0.01
0.015
0.02
0.05
0.1
0.15
0.2
C.8 ADAPTIVE CLASSICAL KERNEL VS QUKERNET
To better demonstrate the adaptability of QuKerNet, we compare the performance of Neural Kernel Network (NKN) and QuKerNet. NKN a adaptive kernel construction method proposed in paper (Sun et al., 2018) We conducted 5 random experiments with the default parameters based on the official code of (Sun et al., 2018), and the results are summarized in the table below. The test accuracy provided in the Table 5. The performance of NKN is lower than our method on all three datasets (i.e.,47 47.6%% vs 94.4% on Tailored CC). Additionally, we observed that the training accuracy of NKN on the three dataset is close 100%, indicating potential overfitting. This could be attributed to two factors: 1) We used default parameters without any optimization, and 2) The limited size of the training samples.
Table 5: The test accuracy for three datasets of different methods.
RBFK
QuKerNet
NKN
Tailored MNIST
83,60 + 2.52
86,80 + 1,36
19.73 + 2.78
Tailored CC
92.80 1 0.40
94.40 + 0.80
47.67 9.23
Synthetic dataset
71.67
91.33 1.63
50.60 5.35
C.9 ACTIVATE LEARNING
Our proposed scheme in QuKerNet offers general framework that can be seamlessly integrated with various advanced methods and techniques. Here, we implement the method of Bayesian optimization (BO) for quantum kernel search using Optuna (Akiba et al., 2019) to show the flexibility of our approach. For the BO, We set the number of iterations to 1000, and the experimental results are shown in the Table 6. The kernel searched by Bayesian optimization did not surpass the random search approach in search space (i.c., 85.6% vs 86.8% on Tailored MNIST), which may be due to the limited number of iterations.
Table 6: BO vs random search using neural predictor.
Training data
Candidate
Fine-tune
Tailored MNIST BO
82.13 = 0.27
83,20
1.66
85,60
1.4
Tailored MNIST Ours
82.13 0.27
85,87
1.43
86,80
1,36
Tailored CC BO
87.00 0.00
89,80
+: 1.72
93.00
1.41
Tailored CC Ours
87
0.00
92.20
+ 1.17
94,40
0.80
Synthetic dutaset BO
80.33 0.67
76.67
6.15
89.34
2.26
Synthetic dataset Ours
80.33 0.67
84.00
1.889
91.33
1.63
1.10 20 QUBITS SIMULATION RESULTS
The experimental results of 20 qubits on three datasets are shown in Table 7. We set M = 50, Ly = 1, k 5, | 5, to condeut three random experiments. From the table, also find that our method works well. Since our method can still search the kernels with good performance on the three datasets (e.g., from 84.89% to 86.45% on Tailored MNIST dataset).
24

---

Published as a conference paper at ICLR 2024
Table 20 qubits simulation results on three datasets.
Training data
Candidate
Fine-tune
Tailored MNIST
84.89 = 0.31
86.45 = 0.32
86.45 :: 0.32
Tailored CC
92.00 1 0.00
92.20 = 0.00
95,00 +: 0.00
Synthetic dataset
70,00 + 0.00
73.33 = 4.71
75.55 = 3.13
C.I1 EXPERIMENTS ON HIGHER-DIMENSIONEE MNIST DATA.
To better demonstrate the effectiveness of our algorithm, increased the dimensionality of the Tailored MNIST dataset to 80, to test our algorithm. The results in Table demonstrate our method still find kernels with better performance higher-dimensional data.
Table 8: 80 dimension simulation results on Tailored MNIST.
RBFK
HEAK
TEK 14.68417.84
QuKerNet 90.27 0.53
87.60 = 0.03
73.33 = 0.00
C.12 TRADE-OFF BETWEEN THE PERFORMANCE GAIN AND OPTIMIZATION OVERIEAAD.
The main trade-off between the performance gain and optimization overhead originated from finding quantum feature map with good performance within a large exponential search space.
We have conducted relevant experiments investigate this trade-off. The main results is that, in most cases, increasing the number of fine-tuning iterations is equivalent to increase the optimization overhead, can lead performance improvements. Detailed results are displayed in the Table 9. We observes that in most of cases when ( is fixed, the performance of the selected kernel increases with an increase (91.8% vs 93% on Tailored CC when [o|=5). Conversely, when & is fixed, the larger the value of [+], the better the performance of the searched kernel (84% vs 91.33% on Synthetic dataset when k=10).
Table 9: Trade-off results three datasets.
[ =
10
20
Tailored MNIST 5
84.53 2.44
86.27
= 1.72
86.27
1.72
86.80
1.36
Tailored M
83.87 1.43
86.33
1.49
86.33
1.49
Tailored CC
91.20 1.47
91.80 + 1.72
93.20
0.75
94.40
Tailored CC =
92.20 + 1.17
93.00 + 1.10
93.40
1.02
94.40
Synthetic dataset le 5
78.67 3,86
87.00
- 4.14
89.00
1.70
89.00 1: 1.70
Synthetic dataset & = 10
84.00 + 3.89
90.00 + 3.16
91.33 + 1.63
91.33 + 1.63
25


Reviews:
[{"review_id": "LMfFMCNuJU", "reviewer": "Reviewer_e6Tr", "summary": "This paper proposes an automated pipeline to design quantum kernels. The main innovation is an optimisation algorithm that solves the joint optimisation of quantum circuit and circuit parameters that produce the quantum feature map. This uses a neural predictor for the circuit architecture performance. The method is benchmarked on toy datasets.", "questions": "- What is M' (number of sampled layouts) used in the experiments?\n- Can you try against better classical baselines to give a sense of how hard these tasks are and quantify the quantum advantage?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Well written, tackling relevant problem of joint quantum architecture and parameter optimisation\n- Novel two stage optimisation algorithm\n- Neural predictor to reduce computational time, evaluation of KTA as good surrogate for accuracy\n- Benchmarks on multiple datasets against multiple baselines, finding good performance", "weaknesses": "- The method is benchmarked against other quantum kernels and RBF. I think that a fairer benchmark would be against classical kernels that can also be adapted to data, such as [https://arxiv.org/abs/1806.04326], and choose similar number of tunable parameters. Also, I did not see how the parameters of the RBF were tuned.\n- I did not see a clear benefit of the neural predictor. For each dataset one needs to train a new predictor, which requires M circuit layouts with their labels. Then M' layouts are sampled randomly and scored by the neural predictor. I assume that $M'\\gg M$. It would be good to see whether sampling M layouts and scoring them with the actual labels would be worse than using the neural predictor. Also, I would expect that an active learning approach like Bayesian optimisation for the architecture search could be more beneficial since it guides the sampling rather than sampling randomly which can be very suboptimal in this large search space.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GiZRbGxMKt", "reviewer": "Reviewer_FSEH", "summary": "The paper proposes QuKerNet, a data-driven method to design quantum feature maps for quantum kernels. QuKerNet uses feature selection techniques to handle high-dimensional data and reduce the vanishing similarity issue. It incorporates a deep neural predictor to efficiently evaluate the performance of various candidate quantum kernels. QuKerNet jointly optimizes the circuit layout and the variational parameters of the quantum feature map, aiming to achieve the global optimal solution for the kernel design problem. QuKerNet demonstrates comparable performance over prior methods on real-world and synthetic datasets, especially for eliminating the kernel concentration issue and identifying the feature map with prediction advantages.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and easy to follow. The paper demonstrates the superiority of QuKerNet over prior methods on various datasets, and shows the potential advantages of quantum kernels designed by QuKerNet for image/fraud classification tasks.\n\n2. The paper employs feature-selection techniques to handle high-dimensional data and mitigate the vanishing similarity issue that affects quantum kernels on near-term quantum devices. Also the authors use a neural network to predict the performance of different candidate quantum kernels based on their circuit layouts and variational parameters, and selects the top ones for fine-tuning, which is sound and novel for enhancement of quantum kernel methods.", "weaknesses": "1. The author mentioned that part of the advantage of the quantum kernel comes from its being computationally hard to evaluate classically. However, the quantum kernel proposed in this paper relies on an explicit quantum feature map, meaning explicit quantum gates and quantum circuit encoding. From my view, on NISQ devices with fewer qubits and shallower quantum circuits, such a quantum feature map is not computationally hard to evaluate classically. Therefore, the reviewer believes that the quantum advantage of the proposed quantum feature map in this paper is not readily apparent. Providing a theoretical explanation rather than solely relying on empirical demonstration would make the argument more convincing.\n\n2. In the field of quantum machine learning, optimizing both the circuit structure and the parameters of variational quantum gates for specific tasks has been previously studied. Papers [1] and [2] introduced a completely differentiable quantum architecture search (QAS) framework, avoiding the vast circuit structure search space, which the reviewer considers one of the issues faced by the proposed QuKerNet in this paper. \n\n3. While transfer some ideas of QAS to quantum kernel methods can be considered one of the contributions of this paper, the reviewer believes the methods of QAS utilized in the paper are somewhat outdated and lack timeliness. For instance, methods based on reinforcement learning [3] and sample-free approaches [1,2] might enable the model to learn better quantum feature maps.\n\n4. In the experimental settings of this paper, the number of utilized quantum bits seems somewhat limited (up to a maximum of 8 qubits). While the reviewer acknowledges that simulating computing overheads on a moderately sized number of qubits can be computationally intensive (even supercomputers struggle to simulate just over 30 qubits in full amplitude simulation), it is believed that the use of only 8 qubits is insufficient to demonstrate the computational gap between quantum and classical kernels. The reviewer is pleased to see more experimental results on a larger number of qubits.\n\n[1] Wu W, Yan G, Lu X, et al. QuantumDARTS: Differentiable Quantum Architecture Search for Variational Quantum Algorithms. ICML, 2023.\n\n[2] Lu X, Pan K, Yan G, et al. QAS-Bench: Rethinking Quantum Architecture Search and A Benchmark. ICML, 2023.\n\n[3] Ostaszewski M, Trenkwalder L M, Masarczyk W, et al. Reinforcement learning for optimization of variational quantum circuit architectures. NeurIPS, 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dQ8c04KCo9", "reviewer": "Reviewer_oLos", "summary": "This work exploits a discrete-continuous joint optimization framework to enhance the quantum kernel, namely QuKerNet, for machine learning problems. The QuKerNet enables the automatic design of kernels by jointly both circuit layouts and variational parameters. Moreover, the authors devise a surrogate loss to efficiently optimize QuKerNet.", "questions": "1. Why does the quantum neural architecture search a key topic? \n\n2. Since the two-stage alternative optimization has to seek a locally optimal quantum architecture, the computational cost is significantly high. So, what is the trade-off between the performance gain and optimization overhead? Is it necessary to seek an optimal quantum neural architecture?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors put forth a QuKetNet to enhance the quantum kernel, where a joint discrete-continuous optimization framework is expected to resolve. \n\n2. A new two-stage alternative optimization algorithm is proposed to find both the best quantum neural architecture and optimal model parameters. \n\n3. Comprehensive numerical simulations are conducted.", "weaknesses": "1. Since a randomized quantum kernel has attained outstanding quantum representations and exactly corresponds to a special quantum neural network, the motivation of using an enhanced quantum kernel with automatic neural architecture search is not a key topic. \n\n2. The computation cost of the two-stage optimization for the objective as Eq. (4) is such high that only a small subset of MNIST samples is tested for the experiments. However, the randomized quantum kernel can be simply incorporated into a quantum kernel learning architecture, which can be extended to large datasets. \n\n3. The performance gain in a selected MNIST data subset cannot be convincing enough to demonstrate the effectiveness of the proposed enhanced quantum kernel. \n\n4. For the empirical simulation, many hyper-parameter settings are configured, which can be particularly fitted to certain subsets but difficult to generalize to all data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xi5VQ5qgLl", "reviewer": "Reviewer_3hUP", "summary": "The authors propose a neural auto-designer to automatically select quantum kernels and update the internal rotation parameters. However, the problem is only the subset of well-studied Quantum Architecture Search (QAS). The proposed method is highly similar to previous literature [1] (which is not even cited in this paper). Results on three datasets are provided with the results slightly better than the selected baselines. However, the numerical experiments are extremely weak with simple datasets and naive baselines.\n\n[1]  Quantum circuit architecture search for variational quantum algorithms.", "questions": "I have no further questions.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "The overall structure of this paper is ok.", "weaknesses": "**The authors are twisting some basic ideas and missing some critical references.** \n1. The problem of quantum kernel auto-designer defined by the authors is only a special case of QAS. QAS has attracted continuous attention with numerous algorithms trying to solve this problem (see a small survey on this problem [1]). The state-of-the-art method [2] on QAS has thoroughly examined their algorithm on both VQE problems and quantum kernels (using MNIST as well). It is unclear why we should focus on only the quantum kernels instead of the general QAS problem.\n2. [3] proposed a very similar QAS approach compared to this paper and [3] is not even mentioned in this paper. It is highly suspicious since the authors cite 7 other papers from the group (or more specifically from Yuxuan Du). The proposed approach is too simple with no novelty at all.\n3. The experimental results are weak and lack a proper comparison. Not a single baseline from all the QAS methods is unacceptable since [2,3] and many other QAS papers all take quantum kernels as one of the testbeds for their QAS approaches. The authors constantly mentioned that the work aims to \"greatly improve the power of quantum kernels under NISQ setting\" but they are not considering noise models or quantum devices.\n\n\n[1] QAS-Bench: Rethinking Quantum Architecture Search and A Benchmark\n\n[2] QuantumDARTS: Differentiable Quantum Architecture Search for Variational Quantum Algorithms\n\n[3] Quantum circuit architecture search for variational quantum algorithms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)