Concentration of Data Encoding in Parameterized Quantum Circuits
Abstract
Variational quantum algorithms have been acknowledged as the leading strategy to realize near-term quantum advantages in meaningful tasks, including machine learning and optimization.
When applied to tasks involving classical data, such algorithms generally begin with data encoding circuits and train quantum neural networks (QNNs) to minimize target functions.
Although QNNs have been widely studied to improve these algorithms’ performance on practical tasks, there is a gap in systematically understanding the influence of data encoding on the eventual performance.
In this paper, we make progress in filling this gap by considering the common data encoding strategies based on parameterized quantum circuits.
We prove that, under reasonable assumptions, the distance between the average encoded state and the maximally mixed state could be explicitly upper-bounded with respect to the width and depth of the encoding circuit.
This result in particular implies that the average encoded state will concentrate on the maximally mixed state at an exponential speed on depth.
Such concentration seriously limits the capabilities of quantum classifiers, and strictly restricts the distinguishability of encoded states from a quantum information perspective.
To support our findings, we numerically verify these results on both synthetic and public data sets.
Our results highlight the significance of quantum data encoding and may shed light on the future design of quantum encoding strategies.
Introduction
Quantum machine learning [1, 2, 3, 4] is an emerging and promising interdisciplinary research direction in the fields of quantum computing and artificial intelligence.
In this area, quantum computers are expected to enhance machine learning algorithms through their inherent parallel characteristics, thus demonstrating quantum advantages over classical algorithms [5].
With the increasing enormous efforts from academia and industry the current quantum devices (usually acknowledged as the noisy intermediate-scale quantum (NISQ) devices [6]) already can show quantum advantages on certain carefully designed tasks [7, 8] despite their limitations in quantum circuit width and depth.
A timely direction is to explore quantum advantages with near-term quantum devices in practical machine learning tasks and a leading strategy is the hybrid quantumclassical algorithms, also known as the variational quantum algorithms [9, 10].
Such algorithms usually use a classical optimizer to train quantum neural networks (QNNs).
They in particular hand over relatively intractable tasks to quantum computers while those rather ordinary tasks to classical computers.
[Figure] For usual quantum machine learning tasks, a quantum circuit used in the variational quantum algorithms is generally divided into two parts: a data encoding circuit and a QNN.
On the one hand, developing various QNN architectures is the most popular way to improve these algorithms’ ability to deal with practical tasks.
Numerous architectures such as strongly entangling circuit architectures [11], quantum convolutional neural networks [12], tree-tensor networks [13], and even automatically searched architectures [14, 15, 16, 17] have been proposed.
On the other hand, one has to carefully design the encoding circuit, which could significantly influence the generalization performance of these algorithms [18, 19].
Consider an extreme case.
If all classical inputs are encoded into the same quantum state, these algorithms will fail to do any machine learning tasks.
In addition, the kernel’s perspective [20, 21, 22, 23] also suggests that data encoding strategy plays a vital or even leading role in quantum machine learning algorithms [24, 25, 26].
However, there is much less literature on data encoding strategies, which urgently requires to be studied.
Encoding classical information into quantum data is nontrivial [1], and it is even more difficult on near-term quantum devices.
One of the most feasible and popular encoding strategies on NISQ devices is based on parameterized quantum circuits (PQCs) [27], such as the empirically designed angle encoding [24, 28], IQP encoding [29], etc.
It is natural to ask how to choose these encoding strategies and whether there are theoretical guarantees of using them.
More specifically, it is necessary to systematically understand the impact of such PQC-based encoding strategies on the performance of QNNs in practical quantum machine learning tasks.
In this work, we present results from the perspective of average encoded quantum state with respect to the width and depth of the encoding PQCs.
A cartoon illustration summarizing our main result is depicted in Fig.
We show that for the usual PQC-based encoding strategies with a fixed width, the average encoded state is close to the maximally mixed state at an exponential speed on depth.
The contributions of this paper are multi-fold:
• We theoretically give the upper bound of the quantum divergence between the average encoded state and the maximally mixed state, which depends explicitly on the hyperparameters (e
g., qubit number and encoding depth) of PQCs.
From this bound, we find that for a fixed qubit number, the average encoded state concentrates on the maximally mixed state exponentially on the encoding depth.
• We show that the quantum states encoded by deep PQCs will seriously limit the trainability of a quantum classifier and further limit its classification ability.
• We show that the quantum states encoded by deep PQCs are indistinguishable from a quantum information perspective.
• We support the above findings by numerical experiments on both synthetic and public data sets.
Related Work Ref.
[18] derived generalization bounds of PQC-based data encoding, which mainly depends on the total number of circuit gates.
While we derive quantum divergence bound that depends on the width and depth of PQCs.
The works [26, 30] explored the effects of data encoding from the perspective of data re-uploading.
[31] studied the robustness of data encoding for quantum classifiers. Data encoding strategies with discrete features were proposed for variational quantum classifiers [32].
1.1 Background
Quantum Basics To better understand this paper, some basic concepts about quantum computing [33] are provided here.
In general, quantum information is described by quantum states.
An n -qubit quantum state is mathematically represented by a positive semi-definite matrix (a.k.a.
a density matrix) ρ ∈ C 2 n × 2 n with property Tr
( ρ ) = 1 . If Rank( ρ ) = 1 , it is called a pure state; otherwise, it is a mixed state.
A pure state can also be represented by a unit column vector | ϕ ⟩ ∈ C 2 n , where ρ = | ϕ ⟩⟨ ϕ | and ⟨ ϕ | = | ϕ ⟩ † .
A mixed state can be regarded as a weighted sum of pure states, i.e., ρ = (cid:80) i q i | ϕ
i ⟩⟨ ϕ i | , where q i ≥ 0 and (cid:80) i q i = 1 .
Specifically, a mixed state whose density matrix is proportional to the identity matrix is called the maximally mixed state 1 ≡ I 2 n .
A quantum state ρ could be evolved to another state ρ ′ through a quantum circuit (or gate) mathematically represented by a unitary matrix U , i.e., [-⇢ 
-]ρ ′ =
UρU † .
Typical single-qubit gates include Pauli gates, X ≡ [ 0 1 1 0 ] , Y ≡ [ 0 − i i 0 ] , Z ≡ [ 1 0 0 − 1 ] , and their corresponding rotation gates R P ( θ ) ≡ e − iθP/with a parameter θ and P ∈ { X, Y, Z } .
Another commonly used gate U 3 appeared in this paper is defined as U 3( θ 1 , θ 2 , θ 3 ) ≡ R z ( θ 3 ) R y ( θ 2 ) R
z ( θ 1 ) , which can implement an arbitrary single-qubit unitary transformation with appropriate parameters.
In this paper, R z , R y are equivalent to R Z , R Y without specified.
A multi-qubit gate can be either an individual gate (e.g., CNOT) or a tensor product of single-qubit gates.
To get classical information from quantum state ρ ′ , one needs to perform quantum measurements, e.g., calculating the expectation value ⟨ H ⟩ =
Tr( Hρ ′ ) of a Hermitian matrix
H , and we often call H an observable.
Quantum Divergence Similar to Kullback-Leibler divergence in machine learning, we use quantum divergence to quantify the difference between quantum states or quantum data.
Two widely-used quantum divergences are quantum sandwiched Rényi divergence [34, 35] (cid:101)
D α ( ρ ∥ σ ) ≡
[Equation]
log Tr (cid:104) σ
[Equation] [Equation]
ρσ
[Equation] [Equation] (cid:105) α and the Petz-Rényi divergence [36] D α ( ρ [Equation]
(cid:2)
[Equation]
(cid:3) , where α ∈ (0 , 1) ∪ (1 , ∞ ) and the latter has an operational significance in quantum hypothesis testing [37, 38, 39].
In this work, for the purposes of analyzing quantum encoding, we focus on the
Petz-Rényi divergence with α = 2 , i.e.,
[Equation]
which also plays an important role in training quantum neural networks [40] as well as quantum communication [41].
Throughout this paper, when we mention the quantum divergence, we mean the
Petz-Rényi divergence D 2 if not specified; log denotes log 2 if not specified.
Parameterized Quantum Circuit In general, a parameterized quantum circuit [27] has the form
U ( θ ) = (cid:81)
j U j ( θ j ) V j , where θ is its parameter vector, U j ( θ j ) =
e − iθ j P j / 2 with P j denoting a Pauli gate, and V j denotes a fixed gate such as Identity, CNOT and so on.
In this paper, PQCs are utilized as both data encoding strategies and quantum neural networks.
Specifically, when used for data encoding, an n -qubit PQC takes a classical input vector x as its parameters and acts on an initial [-state| 
 i ⌦-]state | 0 ⟩ ⊗ n to obtain the encoded state | x ⟩ .
Here, | [-
 i ⌦-]0 ⟩ ⊗ n is a 2 n -dimensional vector whose first element is 1 and all other elements are 0 .
Main Results 2.1 A Warm-up Case
For a quick access, we first consider one of the most straightforward PQC-based data encoding strategies, i.e., consisting of R y rotations only, cf.
Fig.
2(a).
It can be viewed as a generalized angle encoding.
For a classical input vector x with nD components, the output of this data encoding circuit is a pure state | x ⟩ ∈ C 2 n expanded in a 2 n -dimensional Hilbert space.
We denote the density matrix of the output state by ρ ( x ) =
| x ⟩⟨ x | .
If we assume each element of the input vector obeys an independent Gaussian distribution (IGD, see Fig. 2(b) for an intuitive illustration), then we have the following theorem.
Theorem 1.
Assume each element of an nD -dimensional vector x obeys an IGD, i.e., x j,d ∼ N ( µ j,d , σ 2 j,d ) , where σ j,d ≥ σ for some constant σ and 1 ≤ j ≤ n, 1 ≤ d ≤ D .
If x is encoded into
| [-
 i-]0 ⟩
R y ( x 1 , 1 )
R y ( x 1 , 2 ) · · ·
R y ( x 1 ,D ) | [-
-]0 ⟩
R y ( x 2 , 1 )
R y ( x 2 , 2 ) · · ·
R y ( x 2 ,D ) · · · · · · · · · | [-
 i-]0 ⟩
R y ( x n, 1 )
R y ( x n, 2 ) · · ·
R y ( x n,D ) (
[Figure]
Circuit with R
[Figure]
rotations only ππ 3 π2 π x t x j x 4 xx 2 x ππ 3 π2 π y t
{+y j y 
 yy 
 y+}
[Figure]
Note an n -qubit pure state ρ ( x ) according to the circuit in Fig. 2(a), then the quantum divergence between the average encoded state ¯ ρ and the maximally mixed state 1 is upper-bounded as
[Equation]
(2) where ¯ ρ is defined as ¯ ρ ≡ E [ ρ ( x )] . This theorem shows that the upper bound of the quantum divergence between ¯ ρ and 1 explicitly depends on the qubit number n and the encoding depth D under certain conditions.
Further by approximating Eq.
(2) as
[Equation]
we easily find that for a fixed n , the upper bound decays exponentially with D growing in Ω( poly log( n )) .
This means that the average encoded state will quickly approach the maximally mixed state with an arbitrarily small distance under reasonable depths.
Sketch of Proof.
For the j -th qubit, let x j, sum = (cid:80) d x j,d and µ j, sum = (cid:80) d µ j,d , we get x j, sum ∼ N ( µ j, sum , (cid:80) d
σ 2 j,d ) .
From the fact that R y ( x 1 ) R y ( x 2 ) =
R y
( x 1 + x 2 )
, we have ρ ( x j ) =
R y ( x j, sum ) | 0 ⟩⟨ 0 | R † y ( x j, sum ) =
12 [ I + cos ( x j, sum )
Z
+ sin ( x j, sum ) X ] .
Further from the fact that if a variable x ∼ N ( µ, σ 2 ) , then E [cos( x )] = e − σ 2 2 cos( µ ) and E [sin( x )] =
e − σ 2 2 sin( µ ) , together with the condition σ j,d ≥ σ , we bound D 2 ( E [ ρ ( x j )] ∥ 1 ) via calculating
[Equation]
Finally we generalize Eq.
(4) to multi-qubit cases, and complete the proof of Theorem 1.
The detailed proof is deferred to Appendix A.
2.2 General Case
Next, we consider the general PQC-based data encoding strategies shown in Fig.
3, where a column of U 3 gates and a column of entangled gates spread out alternately.
Theorem 2.
(Data Encoding Concentration) Assume each element of a 3 nD -dimensional vector x obeys an IGD, i.e., x j,d,k ∼ N ( µ j,d,k , σ 2 j,d,k ) , where σ j,d,k ≥ σ for some constant σ and 1 ≤ j ≤ n, 1 ≤ d ≤ D, 1 ≤ k ≤ 3 .
If x is encoded into an n -qubit pure state ρ ( x ) according to the circuit in
Fig.
3, the quantum divergence between the average encoded state ¯ ρ and the maximally mixed stateis upper-bounded as
[Equation]
| [-
 i-]0 ⟩
U 3( x 1 , 1 )
Etg
U 3( x 1 , 2 )
Etg· · ·
U 3( x 1 ,D − 1 )
Etg D −
U 3( x 1 ,D ) | [-
-]0 ⟩
U 3( x 2 , 1 )
U 3( x 2 , 2 ) · · ·
U 3( x 2 ,D − 1 )
U 3( x 2 ,D ) · · · · · · · · · · · · | [-
-]0 ⟩
U 3( x n, 1 )
U 3( x n, 2 ) · · ·
U 3( x n,D − 1 ) U 3( x n,D )
This theorem shows that, when employing general PQC-based encoding strategies, we could also have an upper bound of the quantum divergence D 2 (¯ ρ ∥ 1 ) which explicitly depends on n and D . By approximating the upper bound in Eq. (5) as follows
[Equation]
we observe similarly that for some fixed n , the upper bound decays at an exponential speed as D grows in Ω( poly ( n )) .
In addition, according to our proof analysis, even if each U 3 gate is replaced by a U 2 gate containing only two different kinds of Pauli rotations or even a U 1 gate with only one proper Pauli rotation, we still get a similar bound as Eq.
Therefore, we conclude that as long as
D grows within a reasonable scope, the average of the quantum states encoded by a wide family of
PQCs will quickly concentrates on the maximally mixed state.
Unlike the warm-up case, the proof for this theorem is quite non-straightforward due to the tricky entangled gates.
For smooth reading, we defer the complete proof in Appendix B.
The average encoded state ¯ ρ in Theorem 2 is built on infinite data samples, but in practice we do not have infinite ones.
Therefore, we provide the following helpful corollary.
Corollary 2.1.
Assume there are M classical vectors { x ( m ) } Mm =1 sampled from the distributions described in Theorem 2 and define ¯ ρ M ≡ 1 M (cid:80) Mm =1 ρ ( x ( m ) ) .
Let H be a Hermitian matrix with its eigenvalues ranging in [ − 1 , 1] , then given an arbitrary ϵ ∈ (0 , 1) , as long as the encoding depth
D ≥ 1 σ 2 [( n + 4) ln 2 + 2 ln (1 /ϵ )] , we have
[Equation]
with a probability of at least 1 − 2e − Mϵ 2 / 8 .
This corollary implies that for a reasonable encoding depth D and number of samples M , the practical average encoded state ¯ ρ M will also be infinitely close to the maximally mixed state with a high probability.
The proof is mainly derived from Hoeffding’s inequality [42] and the relations between quantum divergence and trace norm.
The proof details are deferred to Appendix C.
Applications in Quantum Supervised Learning
In this section, we will show that the concentrated quantum states encoded by the above PQC-based data encoding strategies will severely limit the performances of quantum supervised learning tasks.
Before beginning, let’s define the following necessary data set.
Definition 3.
(Data Set)
The K -class data set D ≡ { ( x ( m ) , y ( m ) ) } KM m =1 ⊂ R 3 nD × R K totally has
KM data samples, including M samples in each category.
Here, suppose elements in the same entry of all input vectors from the same category are sampled from the same IGD with a variance of at least σ 2 and each x ( m ) is encoded into the corresponding pure state ρ ( x ( m ) ) according to the circuit in Fig.
3 with n qubits and D layers of U 3 gates.
The label y ( m ) is a one-hot vector that indicates which of the K classes x ( m ) belongs [-to.3.
-]to.
3.1 In the Scenario of Quantum Classification
Quantum classification, as one of the most significant branches in quantum machine learning, is widely studied nowadays, where quantum-enhanced classification models are expected to achieve quantum advantages against the classical ones in solving classification tasks [29, 43].
Specifically, in the NISQ era, plenty of variational quantum classifiers based on parameterized quantum circuits are developed to make full use of NISQ devices [9, 44, 11, 12, 13, 45, 46].
See [47] for a review on the recent advances of quantum classifiers.
In general, a quantum classifier aims to learn a map from input to label by optimizing a loss function constructed through QNNs to predict the label of an unseen input as accurately as possible.
Now, we demonstrate the performance of a quantum classifier on the data set D defined in Def. 3.
In this paper, the loss function is defined from the cross-entropy loss with softmax function [48]:
[Equation]
L
[Equation] m ) [Equation] where y ( m ) k denotes the k -th element of the label y ( m ) and
[Equation]
which means the Hermitian operator H k is finally measured after the quantum neural network U ( θ ) .
Here, each H k is chosen from tensor products of various Pauli matrices, such as Z ⊗ I , X ⊗ Y ⊗ Z and so on.
By minimizing the loss function with a gradient descent method, we could obtain the final trained model U ( θ ∗ ) with the optimal or sub-optimal parameters θ ∗ .
After that, when provided a new input quantum state ρ ( x ′ ) , we compute each h [-
-]′ k with parameters θ ∗ according to Eq.
(9), and the index of the largest h ′ k is exactly our designated label.
However, all these graceful expectations can only be established on gradients with relatively large absolute values.
On the contrary, gradients with significantly small absolute values will cause a severe training problem, for example the barren plateau issue [49].
Therefore, in the following we investigate the partial gradient of the cost defined in Eq.
(8) with regard to its parameters.
The results are exhibited in Proposition 4.
Proposition 4.
Consider a K -classification task with the data set D defined in Def. 3.
If the encoding depth D ≥ 1 σ 2 [( n + 4) ln 2 + 2 ln (1 /ϵ )] for some ϵ ∈ (0 , 1) , then the partial gradient of the loss function defined in Eq.
(8) with respect to each parameter θ i of the employed QNN is bounded as
[Equation]
with a probability of at least 1 − 2e − Mϵ 2 / 8 .
From this proposition, we observe that no matter what QNN structures are selected, the absolute gradient value can be arbitrarily small with a very high probability for the above data set D , provided that the encoding depth D and the number of data samples M are sufficiently large.
This vanishing of the gradients will severely restrict the trainability of QNNs.
Moreover, if before training U ( θ ) is initialized to satisfy a certain randomness, such as unitary 2-design [50], then each h k in Eq.
(9) will concentrate on 0 with a high probability, thus the loss in Eq.
(8) will concentrate on ln K .
This concentration of loss is also verified through numerical simulations, as presented in Sec. 4.
This phenomenon, together with Proposition 4, implies that large encoding depth will significantly hinder the training of a quantum classifier and probably lead to poor classification accuracy.
Please refer to
Appendix D for the proof of Proposition 4.
3.2 In the Scenario of Quantum State Discrimination
Quantum state discrimination [51] is a central information-theoretic task and finds applications in various topics such as quantum cryptography [52], quantum error mitigation [53], and quantum data hiding [54].
It aims to distinguish quantum states using a positive operator-valued measure (POVM), a set of positive semi-definite operators that sum to the identity operator.
Here, we have to seriously note that in quantum state discrimination, we can only measure each quantum state once, instead of measuring repeatedly and calculating the expectations as shown in quantum classification. | 0 ⟩
R y ( x 1 ) • R y ( x 4 d +1 ) · · · U 3( θ 1 , 5 , 9 ) • R y ( θ 4 l +9 ) ⟨ H k ⟩ | 0 ⟩ R y ( x 2 ) •
R y ( x 4 d +2 ) · · · U 3( θ 2 , 6 , 10 ) • R y ( θ 4 l +10 ) | 0 ⟩ R y ( x 3 ) • R y ( x 4 d +3 ) · · · U 3( θ 3 , 7 , 11 ) •
R y ( θ 4 l +11 ) | 0 ⟩ R y ( x 4 ) • R y ( x 4 d +4 ) · · · U 3( θ 4 , 8 , 12 ) • R y ( θ 4 l +12 ) ×
( D − 1) × L QNN
In general, a perfect discrimination (i.e., a perfect POVM) can not be achieved if quantum states are non-orthogonal.
A natural alternative option is by adopting some metrics such as the success probability so that the optimal POVM could be obtained via various kinds of optimization ways, e.g.,
Helstrom bound [55] and semi-definite programming (SDP) [56].
Recently, researchers also try to train QNNs as substitutions for optimal POVMs [57, 58].
Next, we demonstrate the impact of the encoded quantum states from the data set D defined in Def.on quantum state discrimination.
Our goal is to obtain the maximum success probability p succ by maximizing the success probability over all POVMs with K operators:
[Equation]
with
[Equation]
where y ( m ) k denotes the k -th element of the label y ( m ) and { Π k } Kk =1 denotes a POVM, which satisfies (cid:80)
Kk =1 Π k
= I .
Proposition 5.
Consider a K -class discrimination task with the data set D defined in Def. 3.
If the encoding depth D ≥ 1 σ 2 [( n + 4) ln 2 + 2 ln (1 /ϵ )] for a given ϵ ∈ (0 , 1) , then with a probability of at least 1 − 2e − Mϵ 2 / 8 , the maximum success probability p succ is bounded as
[Equation]
This proposition implies that as long as the encoding depth D and the data numbers M are large enough, the optimal success probability p succ could be arbitrarily close to 1 K with a remarkably high probability for the data set D .
This nearly blind-guessing success probability shows that the concentration of the encoded quantum states in the data set D will lead to the failure of state discrimination via POVM.
As POVMs are the most general kind of measurements one can implement to extract classical information from quantum systems [33], we conclude that the above different classes of encoded states are indistinguishable from the perspective of quantum information.
The proof of Proposition 5 could be derived straightforwardly by combining Eq.
(11) with Corollary 2.1.
Numerical Experiments
Previous sections demonstrate that the average encoded state will concentrate on the maximally mixed state under PQC-based data encoding strategies with large depth.
These encoded states theoretically cannot be utilized to train QNNs or distinguished by POVMs.
In this section, we verify these results on both synthetic and public data sets by choosing a commonly employed strongly entangling circuit [11], which helps to understand the concentration rate intuitively for realistic encoding circuits. All the simulations and optimization loop are implemented via Paddle Quantum 2 on the PaddlePaddle Deep Learning Platform [59]. [Figure]
[Figure] T r a i n i ng L o ss depth = 2 depth = 4 depth =
depth
= 10 loss of ln (c)
4.1 On Synthetic Data Set
Data Set
The synthetic two-class data set (cid:8) ( x ( m ) , y ( m ) )
(cid:9) M m =1 is generated following the distributions depicted in Fig. 2(b), where each x ( m ) j ∼ N ( µ j , σ 2 j ) for 1 ≤ j ≤ t and y ( m ) denotes a one-hot label.
Here, we assume all means come from two lines, i.e., µ j = 2 π 16 ( j − 1) mod 2 π for classand µ j = 2 π 16 (16 − j ) mod 2 π for class 1, and all σ j ’s are set as 0.8.
Note that the same variance is selected for both classes to facilitate the demonstration of the experiment.
Other choices of σ j ’s would have similar effects.
We first verify our two main upper bounds given in Theorems 1 and 2 by encoding the nD -dimensional inputs that belong to the same class into n -qubit quantum states with D encoding depths under the encoding strategies illustrated in Figs. 2(a) and 4, respectively.
Here, n is set as 2 , 4 , 6 and D ∈ [1 , 14] .
The results are displayed in Fig.
5, from which we can intuitively see that the divergences decrease exponentially on depth.
Specifically, from Fig.
5(a), we know the upper bound in Theorem 2(a) is tight, which is also easily verified from our proof.
From Fig.
5(b), we learn that the upper bound in
Theorem 2 is quite loose, which suggests that the real situation is much worse than our theoretical analysis.
We also notice in Fig.
5(b) that for this strongly entangling encoding strategy, the larger the qubit number is, the faster the divergence decreases.
This unexpected phenomenon reveals the possibility that specific structures of encoding circuits may lead to more severe concentrations for larger numbers of qubits and is worthy of further studies. {+8 Depth 1010 - 
 D 2 ( ρ || I / 2 n ) Digit 2 Digit+}
Digit 3 Digit(a)      8 Depth 0.
T e s t
A cc u r ac y classifying 2 and 9 classifying 3 and(b) 50 100 150 200 250 300 Iterations 0. T r a i n i ng L o ss depth = 2 depth = 3 depth =depth = 6 depth = 8 loss of ln (c)
The curves for the quantum divergence between the averaged encoded state ¯ ρ of each handwritten digit and the maximally mixed state 1 decrease exponentially on depth
The test accuracy reduce rapidly with a larger encoding depth
In the case of classifying digits 3 and 6, when the depth is large (e.g., 8), it is difficult to keep the training loss away from ln 2 in the training process
Next, we examine the performance of QNNs and POVMs by generating 20k data samples for training and 4k for testing under the encoding strategy in Fig.
4, where half of the data belong to class 0, and the others belong to class 1.
The QNNs are designed according to the right hand side of Fig.
4, where the number of layers L QNN is set as n + 2 , the finally measured Hermitian operators are set as H [-
-]1 =
Z and H
= X on the first qubit, and all parameters θ are initialized randomly in [0 , 2 π ] .
During the optimization, we adopt the Adam optimizer [60] with a batch size of 200 and a learning rate of 0 .
In the POVM setting, we directly employ semi-definite programming [56] to obtain the maximum success probability P succ on the training data samples.
The results are illustrated in Fig.
We observe that both the test accuracy of the QNNs and the maximum success probability P succ of the
POVMs eventually decay to about 0 .
5 as the encoding depth grows, indicating that the classification abilities of both the QNNs and the POVMs are no better than random guessing.
In addition, the training losses of QNNs in Fig. 6(c) gradually approach ln 2 as the depth grows and finally do not go down anymore during the whole training process, which implies that the concentration of this data set on the maximally mixed state would limit the trainability of QNNs as we predicted in Sec. 3.1.
All these results are in line with our theoretical expectations.
4.2 On Public Data Set
Data Set and Preprocessing The handwritten digit data set MNIST [61] consists of 70k images labeled from ‘0’ to ‘9’, each of which contains 28 × 28 gray scale pixels valued in [0 , 255] .
In order to facilitate encoding, these images are first resized to 4 × 4 and then normalized to values betweenand π .
Finally, we select all images corresponding to two pairs of labels, i.e., (2 , 9) and (3 , 6) , for two binary classification tasks.
For each task, there are about 12k training samples and 2k testing samples, and each category accounts for half or so.
Here we mainly consider the performance of QNN on this data set because POVMs are generally not suitable for prediction.
These 16-dimensional preprocessed images are first encoded into n -qubit quantum states with encoding depth D and then fed into a QNN (cf.
Fig.
4 again).
We set n as 2,3,4,6,8 and D as 8,6,4,3,2 accordingly.
The settings of QNN are almost the same as those used in the synthetic case, except for a new learning rate of 0 . 05 .
From Fig. 7(a), we see that the average state of each digit class concentrates on the maximally mixed state at an approximately exponential speed on depth, which is consistent with our main result.
Furthermore, the outcomes in Figs. 7(b) and 7(c) also confirm the incapability of training of QNNs, provided that the classical inputs are encoded by a higher depth PQC.
Discussion
We have witnessed both theoretically and numerically that for usual PQC-based data encoding strategies with higher depth, the average encoded state concentrates on the maximally mixed state.
We further show that such concentration severely limits the capabilities of quantum classifiers for practical tasks.
Such limitation indicates that we should pay more attention to methods encoding classical data into PQCs in quantum supervised learning.
Our work suggests that the distance between the average encoded state and the maximally mixed state may be a reasonable metric to quantify how well the quantum encoding preserves the features in quantum supervised learning.
The result on the encoding concentration also motivates us to consider how to design PQC-based encoding strategies better to avoid the exponentially decayed distance.
An obvious way this paper implies might be to keep the depth shallow while accompanied by a higher width.
Still, it will render poor generalization performance [19] as well as the notorious barren plateau issue [49].
Therefore, it will be desirable to develop nontrivial quantum encoding strategies to guarantee the effectiveness and efficiency of quantum supervised learning as well as quantum kernel methods [24, 20, 28]. Amplitude encoding [24] may serve as a good candidate, but due to its unavailability on NISQ devices, it is rarely considered or implemented at current stage.
Recent works on data re-uploading [26, 30, 62] and pooling [19, 25] of quantum neural networks may also provide potential methods for improving quantum encoding efficiency.
Acknowledgements
We would like to thank Sanjiang Li, Yuan Feng, Hongshun Yao, and Zhan Yu for helpful discussions. G. L. acknowledges the support from the Baidu-UTS AI Meets Quantum project, the China
Scholarship Council (No. 201806070139), and the Australian Research Council project (Grant No: DP180100691). Part of this work was done when R. Y. and X. Z. were research interns at Baidu
Research. Checklist For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Sec. 2.2.
(c) Did you discuss any potential negative societal impacts of your work? [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results? [Yes] See Fig. 2(b) and the descriptions in every theorem.
(b) Did you include complete proofs of all theoretical results? [Yes] All proofs are included in Appendix. If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [No] Our experimental results could be easily reproduced.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Sec. 4.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[No] All our numerical experiments could be run on a personal laptop.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] See Sec. 4.
(b) Did you mention the license of the assets?
[No] It is publicly available.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage
paid to participants and the total amount spent on participant compensation?
[N/A]
Supplementary Material for “Concentration of Data Encoding in
Parameterized Quantum Circuits”
Theorem 1. Assume each element of an nD -dimensional vector x obeys an IGD, i.e., x j,d ∼ N ( µ j,d , σ 2 j,d ) , where σ j,d ≥ σ for some constant σ and 1 ≤ j ≤ n, 1 ≤ d ≤ D . If x is encoded into an n -qubit pure state ρ ( x ) according to the circuit in Fig.
S1, then the quantum divergence between the average encoded state ¯ ρ = E
[ ρ ( x )] and the maximally mixed state 1 = I 2 n is upper bounded as [Equation] Proof.
Let ρ ( x j ) ≡ R y ( x j, 1 + · · · + x j,D ) | 0 ⟩⟨ 0 | R † y ( x j, 1 + · · · + x j,D ) , then
[Equation]
(S
[Equation]
Due to the independence of each ρ ( x j ) , we have
[Equation] What’s more, for j = 1 , . . . , n [Equation]
Lemma 1.
Assume a variable x ∼ N ( µ, σ 2 ) , then
[Equation]
We know (cid:80) d x j,d ∼ N ( (cid:80) d µ j,d , (cid:113)(cid:80) d σ 2 j,d ) , and combining Eq.
(S5) with Lemma 1, we have (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) E
[ ρ ( x j )] (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) 2 F = 1 2 + 1 2 E
[Equation]
S
[Equation]
Finally, from Eq.
(S3), we have
[Equation]
This completes the proof.
B Proof of Theorem
Theorem 2.
Assume each element of a 3 nD -dimensional vector x obeys an IGD, i.e., x j,d,k ∼ N ( µ j,d,k , σ 2 j,d,k ) , where σ j,d,k ≥ σ for some constant σ and 1 ≤ j ≤ n, 1 ≤ d ≤ D, 1 ≤ k ≤ 3 .
If x is encoded into an n -qubit pure state ρ ( x ) according to the circuit in Fig.
S2, the quantum divergence between the average encoded state ¯ ρ and the maximally mixed state 1 is upper bounded as
[Equation] U 3( x 2 , 1 ) U 3( x 2 , 2 ) · · · U 3( x 2 ,D − 1 )
U 3( x 2 ,D ) · · · · · · · · · · · · | 0 ⟩
U 3( x n, 1 ) U 3( x n, 2 ) · · · U 3( x n,D − 1 ) U 3( x n,D )
Proof.
In this proof, we consider the U 3( x j,d, 1 , x j,d, 2 , x j,d, 3 ) gate as R z ( x j,d, 3 ) · R y ( x j,d, 2 ) · R z ( x j,d, 1 ) , which is one of the most commonly used ones.
Of course, other forms of U3 gate are similar.
Outline of Proof.
1) Decomposing initial state.
Firstly, we decompose the initial state according to Pauli bases; 2) Vectors transition.
Then by taking the corresponding coefficients as a row vector, we state that each action of a group of entangled gates Etg i or a column of U 3 gates is equivalent to multiplying the previous coefficient vector by a transition matrix;
3) Bound by singular value.
Finally, we get the upper bound by investigating the singular values of these transition matrices.
1) Decomposing initial state.
The state after the first column of U 3 gates becomes
[Equation]
[Table]
Now we define ρ 1 ≡ ρ 1 , 1 ⊗ ρ 1 , 2 ⊗ · · · ⊗ ρ 1 ,n , where
[Equation]
[Equation]
S17)
And from Lemma 1, we have
[Equation]
where we define A j,d,k = e − σ 2 j,d,kfor writing convenience.
Here we note that due to all x j,d,k ’s being independent of each other, calculating the expectation of ρ 1 ,j in advance does not affect the following computations.
Next we decompose E [ ρ 1 ,j ] according to the Pauli bases, i.e., I, Z, X, Y ,
[Equation]
Then from Eq.
(S16), we could derive that E [ ρ 1 ] = (cid:78) n j =1 E [ ρ 1 ,j ] can also be decomposed in accordance with various tensor products of Pauli bases.
Therefore, studying the state after the gate
Etg 1 could be transferred to what performance it will be when entangled two-qubit gates act on the tensor products of Pauli bases.
2) Vectors transition.
Here, we focus on the two widely employed two-qubit entangled gates CNOT and CZ, and the calculations are concluded in Table 1.
The results in the table show that the transitions are closed for tensor products of Pauli bases.
Here we note that other entangled two-qubit gates will have a similar effect.
Next, we consider the effects of applying the gate U 3( x 1 , x 2 , x 3 )
= R z ( x 3 ) R y ( x 2 ) R z ( x 1 ) to four Pauli matrices.
And the results of the calculations are as follows: [Equation] where [Equation]
Here, x k , A k , µ k are the abbreviations for x j,d,k , A j,d,k , µ j,d,k , respectively. Now we record Eqs. (S20)-(S23) as a matrix T , which we call transition matrix
, i [Equation] e., [Equation]
By carefully calculating Eqs. (S20)-(S33) again, we also have
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Table]
(cid:35) , (S34) where the three matrices correspond to the effects of applying R z ( x 1 ) , R y ( x 2 ) and R z ( x 3 ) , respectively.
If we further record an arbitrary input ρ in ≡ α 1 I + α 2 Z + α 3 X + α 4 Y as a row vector π in = [ αααα 4 ] , then applying the gate U 3( x 1 , x 2 , x 3 ) to ρ in will result in the output ρ out ≡ β 1 I + β 2 Z + β 3 X + β 4 Y , where
[Equation]
This is a fundamental relationship in this proof, which can be easily verified in multi-qubit and multi-depth cases.
Hence, we could rewrite each E [ ρ d ] , 0 ≤ d ≤ D , as follows
[Equation]
where each T j,d represents that this transition matrix is constructed based on the gate U 3( x j,d, 1 , x j,d, 2 , x j,d, 3 ) and each (cid:103)
Etg i means rearranging the elements of the previously multiplied row vector, which is equivalent to the effect after applying Etg i , 1 ≤ i ≤ D − 1 .
Here, please note that we omit the possible negative sign described in Table 1, because in the following proof, it has no influence.
From the fact that Tr (cid:0) P 2 i (cid:1) = 2 , Tr ( P i P j ) = 0 , where P i , P j denote different Pauli matrices, and combining the relationship in Eq.
(S40), we have
[Equation]
What’s more, we also find that every ⊗ nj =1 T j,d and (cid:103)
Etg i always have an element 1 in the top left corner, i.e.,
[Equation]
where T d , E i ∈ R (4 n − 1) × (4 n − 1) and 1 ≤ d ≤ D, 1 ≤ i ≤ D − 1 .
Therefore,
[Equation]
where ˚ π 0 means that the row vector π 0 removes the first element.
3) Bound by singular value.
Next, in order to further calculate it, we need to prove first the following two Lemmas.
Lemma 2.
Given a Hermitian matrix H ∈ C n × n with all its eigenvalues no larger than λ , and an n -dimensional vector x , then
[Equation]
where ∥ · ∥ 2 denotes the l 2 -norm.
Proof.
Assume H has the spectral decomposition
[Equation]
[Equation]
{+) then x can be uniquely decomposed as x = (cid:80) ni =
+}
α i u i with (cid:80) ni
=1 | α
i | 2 = ∥ x ∥ 22 .
Finally we have x † H x = n (cid:88) i =
[Equation]
n (cid:88)
[Equation]
[Equation]
Lemma 2.
Given a Hermitian matrix H ∈ C n × n with all its eigenvalues no larger than λ , and an arbitrary matrix Q ∈ C n × n with all its singular values no larger than s , then the largest eigenvalue of QHQ † is no larger than s 2 λ .
Proof.
The largest eigenvalue of QHQ † can be computed as λ max ≡ max x x † QHQ † x , where x denotes a unit vector.
Assume Q has the singular value decomposition
Q = USV † = n (cid:88) i =
s i u i v † i = [ uu· · · u n ]   s    s    ...
s n     v †v †... v † n   (S50) and x = (cid:80) n i =
{+α i u i with (cid:80) n i =
+}
| α i | 2 = 1 , then x † QHQ † x
[Equation]
x †
[Equation]
† HV SU †
[Equation]
[Equation]
 
[Equation]

 .
(S51)
Consider V † HV as a new Hermitian matrix and SU † x as a new vector ˜ x , then all the eigenvalues of
V † HV are still no larger than λ and the square of the l 2 -norm of ˜ x is computed as
[Equation]
From Lemma 2, we have
[Equation]
As x is arbitrary, we can obtain that λ max ≡ max x x † QHQ † x is no larger than s 2 λ as well.
Now, let us investigate the singular values of T j,d .
From Eq.
(S33), we know it always has the trivial biggest singular value 1.
The second-biggest singular value s m can be derived from
[Equation]
where u ∈ C 3 denotes a unit column vector.
From Eq.
(S34)
[Equation]
we derive that [Equation] [Figure] [Equation]
[Figure]
[Equation]
[Figure]
[Equation]
hence,
[Equation]
where Q ≡ (cid:20)
[Equation]
(cid:21) has the largest singular value 1.
From Lemma 2, we deduce that the largest eigenvalue of the matrix in Eq.
(S57) is max (cid:8)
A 22 , A 23 (cid:9) , which is no larger than e − σ 2 .
Further combining Eq.
(S54), we infer that s m is no larger than e − σ 2 2 , i.e., the second-biggest singular value of each T j,d is no larger than e − σ 2 2 .
What’s more, we could derive that their tensor product ⊗ nj =1 T j,d also has the trivial largest singular value 1 and the second-largest singular value which is no larger than e − σ 2 2 .
From the definition of T d in Eq.
(S42), we declare that the largest singular value of each T d is no larger than e − σ 2 2 .
Let’s go back to the following formula in Eq.
(S46) to continue estimating
Tr ( E [ ρ D ]) 2 , i.e.,
[Equation]
Since the largest eigenvalue of T D T ⊤ D is no larger than e − σ 2 , and each E i , defined in Eq.
(S42), is a unitary matrix, by repeatedly applying Lemma 2, we obtain that largest eigenvalue of T 1 E 1 · · ·
T D T ⊤ D · · · E ⊤ 1 T ⊤ 1 is no larger than e − Dσ 2 .
Furthermore, from Eq.
(S36) and the definition of ˚ π 0 , we know ˚ π 0 has 4 n − 1 dimensions, where 2 n − 1 elements are 12 n and the others are 0 .
Hence, ∥ ˚ π 0 ∥ 22 = 2 n − 1 2 2 n .
Combining these with Lemma 2, we have
[Equation]
Go further, and we have, together with Eq.
(S46),
[Equation]
Finally, we have
[Equation] This completes the proof of Theorem 2. Without stopping here, we also analyse some generalizations of Theorem 2. (I) From Eqs.
(S54)-(S57)
, we find that removing the matrix Q will have no influence on the final result.
Hence, we directly generalize that if there is only one column of R z R y or R y R z gates in each layer, we will get the same upper bound.
In fact, according to our proof method, we infer that as long as there are two different kinds of rotation gates in each encoding layer, this upper bound is valid.
(II)
What is the result for the case with only R y rotation gates in each encoding layer?
Since each T d has the largest singular value 1, it is not suitable for the above proof.
However, through analyzing the transition rule in Table 1, we find that the largest singular value of T d − 1 E d − 1 T d is still no larger than e − σ 2 2 , which means every two encoding layers have the same effect as above with one layer.
Therefore, the final upper bound can be changed to log (cid:16) 1 + (2 n − 1)e
−⌊ D 2 ⌋ σ 2 (cid:17) .
Since it has the same trend as the original bound, it has no impact on our final analysis.
C Proof of Corollary 2.
Corollary 2.1.
Assume there are M classical vectors { x ( m ) } Mm =1 sampled from the distributions described in Theorem 2 and define ¯ ρ M ≡ 1 M (cid:80) Mm =1 ρ ( x ( m ) ) .
Let H be a Hermitian matrix with its eigenvalues ranging in [ − 1 , 1] , then given an arbitrary ϵ ∈ (0 , 1) , as long as the depth D ≥σ 2 [( n + 4) ln 2 + 2 ln (1 /ϵ )] , we have
[Equation]
with a probability of at least 1 − 2e − Mϵ 2 / 8 .
Proof.
Let ¯ ρ ≡ E (cid:2) ρ (cid:0) x ( m ) (cid:1)(cid:3) , then we have
[Equation]
where the inequality is due to triangle inequality.
Now we first consider the first term in Eq.
(S67).
Since 1 M Tr (cid:0) Hρ ( x (1) ) (cid:1) , . . . , 1 M Tr (cid:0) Hρ ( x ( M ) ) (cid:1) are i.i.d. and − 1 M ≤ 1 M Tr (cid:0)
Hρ ( x ( m ) ) (cid:1) ≤ 1 M , through Hoeffding’s inequality [42], we have
[Equation]
From the fact that (cid:80) Mm =1 1 M Tr (cid:0) Hρ ( x ( m ) ) (cid:1) =
Tr ( H ¯ ρ M ) and E (cid:2)
Tr (cid:0) Hρ
( x ( m ) ) (cid:1)(cid:3) = Tr ( H ¯ ρ ) , we obtain [Equation]
with a probability of at least 1 − 2e − Mϵ 2 8 .
Next we consider the second term in Eq.
(S67).
Since the eigenvalues of H range in [ − 1 , 1] , we obtain
[Equation]
where ∥ · ∥ tr denotes the trace norm and the second inequality is from the Fuchs–van de Graaf inequalities [63], i.e., 1 − (cid:112) F ( ρ, ρ ′ ) ≤ 12 ∥ ρ − ρ ′ ∥ tr ≤ (cid:112) 1 − F ( ρ, ρ ′ ) .
By combining the upper bound in Theorem 2 with the fact that − log F ( ρ, ρ ′ ) ≤ D 2 ( ρ, ρ ′ )
[36], we have [Equation] where in Eq. (S72) we use the condition D ≥ 1 σ 2 [( n + 4) ln 2 + 2 ln (1 /ϵ )] .
By inserting Eq. (S74) into Eq. (S70), we can get [Equation]
Bringing Eqs. (S69) and (S75) into Eq. (S67), we complete the proof of Corollary 2.1. D Proof of Proposition
Proposition 4.
Consider a K -classification task with the data set D defined in Def. 3.
If the encoding depth D ≥ 1 σ 2 [( n + 4) ln 2 + 2 ln (1 /ϵ )] for some ϵ ∈ (0 , 1) , then the partial gradient of the loss function defined in Eq.
(8) with respect to each parameter θ i of the employed QNN is bounded as
[Equation]
with a probability of at least 1 − 2e − Mϵ 2 / 8 .
Proof.
From the chain rule, we know
[Equation] We first calculate ∂L ( m ) ∂h l as follows ∂L ( m ) [Equation] Since e h l / (cid:80) Kj =1 e h j ∈ (0 , 1) and y ( m ) is one-hot, we can get its upper bound as | ∂L ( m ) ∂h l | ≤ 1 .
Next, from the parameter-shift rule [45], we calculate ∂h l ∂θ i as follows
[Equation]
where θ + π 2 means adding π 2 to θ
i and keeping the others unchanged, and θ − π 2 is similarly defined.
If we define
[Equation]
then together with Eqs.
(S77)-(S79), we could bound the gradient as
[Equation]
Here, it could be easily verified that the eigenvalues of ˜ H l (defined in Eq.
(S80)) range in [ − 1 , 1] and
Tr( ˜ H l ) = 0 . Then from Corollary 2.1, we could bound Eq.
(S84) as Eq.
(S85), i.e., for any ϵ ∈ (0 , 1) , provided that the encoding depth D ≥ 1 σ 2 [( n + 4)
ln 2 + 2 ln (1 /ϵ )] , we have | ∂L ( θ ; D ) ∂θ i | ≤ Kϵ with a probability of at least 1 − 2e − Mϵ 2 / 8 .
This completes the proof of Proposition 4.