QTN-VQC: An End-to-End Learning Framework for Quantum Neural Networks
Authors: Jun Qi
Chao-Han Huck Yang
Pin-Yu Chen
Venue: ICLR 2022 Submitted (2022)
Keywords: quantum neural networks
variational quantum circuits
end-to-end learning framework
tensor-train network

Abstract: The advent of noisy intermediate-scale quantum (NISQ) computers raises a crucial challenge to design quantum neural networks for fully quantum learning tasks. To bridge the gap, this work proposes an end-to-end learning framework named QTN-VQC, by introducing a trainable quantum tensor network (QTN) for quantum embedding on a variational quantum circuit (VQC). The architecture of QTN is composed of a parametric tensor-train network for feature extraction and a tensor product encoding for quantum encoding. We highlight the QTN for quantum embedding in terms of two perspectives: (1) we theoretically characterize QTN by analyzing its representation power of input features; (2) QTN enables an end-to-end parametric model pipeline, namely QTN-VQC, from the generation of quantum embedding to the output measurement. Our experiments on the MNIST dataset demonstrate the advantages of QTN for quantum embedding over other quantum embedding approaches.

Under review as a conference paper at ICLR 2022
QTN-VQC: AN END-TO-END LEARNING FRAME- WORK FOR QUANTUM NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
The advent of noisy intermediate-scale quantum (NISQ) computers raises a cru- cial challenge to design quantum neural networks for fully quantum learning tasks, To bridge the gap, this work proposes end-to-end learning framework named QTN-VQC, by introducing a trainable quantum tensor network (QTN) for quan- embedding
variational quantum eircuit (VQC). The architecture of QTN is composed of a parametric tensor-train network for feature extraction and a ten- sor product encoding for quantum embedding. We highlight the QTN for quan- tum embedding in terms of two perspectives: (I) we theoretically characterize QTN by analyzing its representation power of input features; (2) QTN enables an end-to-end parametric model pipeline, namely QTN-VQC, from the generation of quantum embedding to the output measurement. Our experiments on the MNIST dataset demonstrate the advantages of QTN for quantum embedding over other quantum embedding approaches.
INTRODUCTION
The state-of-the--at machine learning (ML), particularly based on deep neural networks (DNN), has enabled a wide spectrum of successful applications ranging from the everyday deployment of speech recognition (Deng et al.) 2013) and computer vision (Sermanet al. 2014) through to the frontier of scientific research in synthetic biology (Jumper et al.] 2021). Despite rapid theoretical and empirical progress in DNN based regression and classification (Goodfellow al., 2016). DNN training algorithms are computationally expensive for many new scientific applications, such as new drug discovery (Smalley 2017), which requires computational resources that are beyond the computational limits of classical hardwares (Freedman) 2019). Fortunately, the imminent advent of quantum computing devices opens up new possibilities of exploiting quantum machine learning (QML) (Biamonte et al. 20001,/Shuuu al.) 2015); Schuld & Petruccione] 2018 ( [Schuld & Killoran, 2019 Saggio et al. [02 Dunjko 2021) to improve the computational efficiency of ML algorithms in the new scientifie domains.
Although the exploitation of quantum computing devices to carry out QML is still in its initial exploratory stages, the rapid development in quantum hardware has motivated advances in quantum neural networks (QNN) to run in noisy intermediate-scall quantum (NISQ) devices (Preskill) 2018 Huggins et al. 2019 Huang al. 2021 Kandala al., 2017). A NISQ device means that not enough qubits could be spared for quantum error correction, and the imperfect qubits have to be directly used the physical layer. Even though, compromised QNN approach is proposed by employing hybrid quantum-classical models that rely on the optimization of variational quantum circuits (VQC) (Benedetti et al.] [019 Mitarai al., 2018). The resilience of the VQC based models to certain types of quantum noise errors and high flexibility concerming coherence time and gate requirements (McClean et al., 2018) admit many practical implementations of QNN on NISQ devices (Chen et | 2020b] Yang et al. 2021 Du et al., 2020; 2021 Skolik et al.) 2021 Dunjko cet al. 2016 Jerbi et al.) 2022 I Ostaszewski et al. 2021). One notable limitation the current QNN training pipeline is that the quantum embedding is not fully realizable in quantum computer, which may impede the learning of the QNN. Hence, this work proposes QTN-VQC to enable an end-to- end trainable QNN, including data embedding quantum measurements, that easily realizable in guantum devices, where QTN stands for the quantum tensor network (Orús) 2019 Huckle et al. 2013 Biamonte et al. 2017 Murg et al., 2010) for generating quantum embedding.

---

Under review as a conference paper at ICLR 2022
Classical data
10)
51
(0)
Ux 14x)
Hol = lgo()))
10)
Quantum Embedding Generation
Variational Quantum Circuit
Measurement M(lgg(()))
Hx
Hg
Figure 1: An illustration of QNN based on VQC.
Input y
Input y
Dense
TTN
x
(0)
(0)
(0)
Guantum
10)
Quantum Embedding
Ux 0x = l4x)
Embedding
Ux 1x
11xx)
10)
10)
(s) dense layer for dimension reduction
(b) Tensor-Train layer for dimension reduction
Figure 2: Different paradigms for quantum embedding. (a) dense layer is used to generate low- dimensional vector x from a high-dimensional one y; (b) a TTN is used for dimension reduction.
As shown in Figure D our QNN builds a unitary linear operator that consists of three main com- ponents: (1) quantum embedding generation; (2) variational quantum circuit; (3) measurement. Quantum embedding generation, also known as quantum encoding, applies fixed unitary linear operator H, transforming classical vectors x to quantum states [xx)
Hilbert space. This step is an important aspect of designing quantum algorithms that directly impact the entire computation cost of VQC and owns characteristic of quantum superposition. Moreover, the VQC comprises two types of quantum gates: (1) Controlled-NOTT (CNOT) gates; (2) learnable parametric quantum gates. The CNOT gates ensure the property of quantum entanglement through mutually connecting the qubits, and the parametric quantum gates be adjustable to best fit the quantum input states. The model parameters of VQC should be optimized by employing variants gradient descent al- gorithms during the training process. Those parametric quantum gates of VQC are similar to the weights assigned to DNN, and such quantum circuits have been justified to be resilient to quantum noises (Farhi al.] 2014 Kandala et 2017 McClean al. 2016). Besides, the measurement M( x))) aims at projecting the quantum output states |ga
to one classical output z;.
This work focuses on quantum embedding generation because it quite related to the practical usage in machine learning applications in terms of computational cost and representation capability of classical input features. particular, we design novel quantum tensor network (QTN) for quantum embedding generation. More specifically, the QTN consists of tensor-train network (TTN) for dimension reduction and quantum tensor encoding framework for outputting quantum embeddings. The dimension reduction is a necessary procedure before the quantum encoding because only small number of qubits could be supported on available NISQ computers at this moment. typical dense
approach for dimension reduction relies layer, to convert nigh-dimensional input vectors into low-dimensional ones x. However, since a dense layer cannot be physically mapped on quantum computer, much overhead has to be incurred by frequently communicating between classical and quantum devices during the end- -nd training pipeline.
classical fully-connected layer, also known
2

---

Under review as a conference paper at ICLR 2022
As shown in Figure 2 (b), one of our contribution is to leverage tensor train network (TTN) to replace the dense layer in Figure 2 (1) TTN can maintain the representation power of the dense layer, which will be justified in our theorems; (2) TTN is a tensor network and can be flexibly placed in quantum computers, which enables end-to-end training process fully conducted in quantum computer. Moreover, in this work, a tensor product encoding (TPE) is delicately designed for generating quantum embedding, which builds the relationship between a classical vector x and the corresponding quantum state [x); Besides, we further investigate the representation of QTN-VQQ in terms of model size and non- linear activation function used in TTN. We denote a QTN as the combination of TTN and TPE and utilize QTN-VQC as a genuine end-to-end learning framework for QNN.
(a). The benefits of applying TTN arise from two aspects:
2 RELATED WORK
The work (Schuld& & Petruccione 2018; Biamonte et al. 2017 (017/DDmjjoo & Briegel 2018) demonstrate that VQC shows great promise in surpassing the performance of classical ML. Prominent examples of VQC based models include quantum approximate optimization algorithm (QAOA) (Farhi et al.]
2014), and quantum circuit learning (QCL) (Mitarai al. 2018). Various architectures and geome- tries of VQC have been shown in tasks ranging from image classification (Henderson et al., 2020
Chen et al.
2020a Kerenidis et al. 2020 to reinforcement learning (Chen et al. 2020b).
As for quantum embedding, basis encoding is the process of associating classical input data in the form of binary strings with the computational basis state of a quantum system (Leymann & Barzen 2020). Similarly, amplitude encoding is technique involving encoding data into the amplitudes of quantum state (Soklakov & Schack 2006). Unfortunately, the computational cost of both quantum embedding and amplitude encoding becomes exponentially expensive with the increasing number of qubits (Schuld & Killoran| 2019). new technique of angle embedding makes use of the quantum gates to generate quantum states (Fu et al.] 2011), but it cannot deal with the high-dimensional feature inputs. Therefore, this work exploits the use of TTN for dimension reduction followed by a TPE for generating quantum embedding.
In particular, this work employs the TTN for dimensionality reduction. The TTN model based on TT decomposition in neural networks was first proposed in (Oseledets 2011). and it could be flexibly extended the convolutional neural network (CNN) (Garipov et al. 20 and recurrent neural network (RNN) TJandra et al.] 2017). The empirical study of TTN on machine learning tasks shows that TTN is capable of maintaining the DNN baseline results (Qi et al.) 2020a Yu et al.] 2017 Yang et al. 20177] Jin al. 2020). However, our best knowledge, no existing works have applied TTN to QML. Besides, since the tensor network- -based machine learning model like TTN is closely related to quantum machine learning in terms of their model structures (Liu & Wang 2018
Gao et al. 2017). the QTN-VQC model can be directly regarded as the classical simulation of the classical dense layer, more complicated architectures like AlexNet [Looyd et al. 2020) could be used for dimension reduction, and we also compare the performance between TTN and Alex bet-based models.
corresponding quantum machine learning. In addition
NOTATIONS
We denote R' as I-dimensional real coordinate space, and [ refers to a space of K- order tensors. The symbol W E I xxx represents a K-order multi-dimenssonal tensor in R1xx2xx-IK and the symbols E TR. and W E RRXxJ represent vector and a matrix, respectively.
For the notations of quantum computing, V R', the symbol [v) denotes a quantum state associ- ated with a 2d -dmeensional vector in a Hilbert space. Particularly, j0) [1 oj² and [1) = [0 1J".
The quantum gate Ry (0) means a Pauli-Y gate with a unitary operator as defined in Eq. 1 which implies qubit rotates the Bloch sphere along the Y-axis by given angle 0.
cos -is
-isin ge
(y
(1)
cos
Moreover, the operator @ is a tensor product. Given the vectors v; E R', the tensor product of vectors is defined as so! Vi, which is 2 dimensioonl vector and can provide compact represen-
3

---

Under review as a conference paper at ICLR 2022
J1
J2
J3
K-2
J
K-1
J
10)
Ry(ox))
12
K-2
K-1
K
10)
Ry(1x)))
W22
Wg
WK-2
Wx-1
Wk
13
K-2
K-1
K
10)
Ry(ox
(6) Tensor-Tral Network: given set of TT-ranks (rn fg..., fgl. circie represents
[b] Tansor Product Encoding: Ry(ax) denates the Pauli Y rotation gate
dimension.
Figure 3: A demonstration of quantum tensor network for quantum embedding.
tation for V1 V2 00...... @ Vy. Similarly, the symbol j0)es @ means a tensor product of S quantum states of (0). Furthermore, for scalar the quantum state u) can be written
cos sin
(e) cos v + sin w/1)
QTN-VQC: OUR PROPOSED END-TO-END LEARNING FRAMEWORK
This section introduces our proposed end-to-end learning framework, namely QTN-VQC in this work. As shown in Figure the QTN model includes two components (a) TTN and (b) TPE, which will be separately introduced in Section 4.1 and Section 4.2 Moreover, Figure illustrates the framework of VQC and Section 4.3 is devoted to discussing the details of VQC.
4.1 TENSOR TRAIN NETWORK FOR DIMENSION REDUCTION
We leverage TTN Novikov et al. 2015 for the dimension reduction input features. TTN re- lies the TT decomposition (Oseledets) (011) and has been commonly employed in machine learning tasks like speech processing (Qi et al. 20205) and computer vision (Yang et al. 2017). The TT decomposition assumes that given W E 12/x12x x |x is factorized into the multiplication of 3-order tensors X₂ E R 1ee-1x1ax0 1x4 In more detail, given set of indices (2,,42,.... ik) X11,, i2, ix) is decomposed as:
of TT-ranks (No, fi, ), K-order tensor
K
X'1 (2.,
(k)
II Xx(1)).
(3)
where Vig [T,], (ile)) E RT 1x7k, Since To 1, the term 1 (e(le) is scalar value.
TTN employs the TT decomposition in dense layer and is explicitly demonstrated in Figure 3 (a). In more detail, for an input tensor X' E 124x4xxxxxx and output tensor y R. we achieve
I
1x w ((i..))) (12. ... (iK. X(1,, (2,,;
=
$
ix
f2
1x
K
II (aleri)u))
II Xal(a)
ix
(i
Pa(jk):

---

Under review as a conference paper at ICLR 2022
where Xx (it) E P and Pi( E R which results in scalar IT (ik) be- cause of the ranks To = r1 = 1: (((11.11) (22..2)..... (ix.jk) jik) is closely associated with W(m,, M2, ..... mx) defined in Eq. (3), if each index m, ig Jg. 18 set. The muulti-iimensionll tensor W is decomposed into the multiplication of 4-order tensors Wy E 1R Xrr,, A non- linear activation function, e.g., Sigmoid, Tanh, and ReLU, is imposed upon the tensor y. Compared with a dense layer with Il J parameters, a TTN owns as few as ΣK P trainable parameters.
When a TTN is utilized for the dimension reduction, the high-dimensional input vector x E R is first reshaped into tensor X' E 124x1xx-xxxx and then we can represent A' as TT format that goes through TTN. The outputs of TTN can be converted back to a tensor y E 1 1g-1xxxx---JJr which is further reshaped to a lower dimensional vector y E Ry_ Here, we define II Ix = / and
IT J/ J. Moreover, the computational complexities of TTN and the related dense layer are in the same scale, which is discussed in Yangg et al. 2017).
Eq. 4 suggests that TTN is multi-dimennsonal extension of dense layer, where the trainable weight matrix of a dense layer is changed to the learnable core tensors. Additionally, many empirical studies demonstrate that TTN is capable of maintaining the baseline results of the dense layer (Qi
itt al. 2020b; Yang et al. 2017 Novikov et al., 2015; Oi et al.] (2020a). More significantly, since TTN can be flexibly mapped into a quantum eircuit, the quantumness inherent in TTN brings great advantages over other architectures like the dense layer. In other words, although TTN is treated classically, it possible to substitute equivalent quantum circuits for TTN when more qubits become available (Du et al. 2020). which implies that QTN-VQC stands for a genuine and-to-end QNN learning architecture on quantum computer.
Furthermore, since the gradient exploding and diminishing problems are serious issues in the TTN training. To avoid those training problems, we only consider 3-order core tensors and small TT-ranks to configure simple TTN in experimental simulations. Our theoretical analysis of QTN-VQC based on Theorem 3 in Section 5 suggests that the representation power is not related to TT-ranks and the tensor order K, thus small TT-ranks and the tensor order K are preferred. In particular, lower K can significantly reduce the computational cost and speed up the convergence rate.
4.2 TENSOR PRODUCT ENCODING
In this subsection, we first introduce Theorem I and then derive our TPE associated with the circuits in Figure (b).
Theorem 1. Given the classical vector = [2,, a.. x1)" E RR, , TPE as shown in Figure 3 (b)
can result in a quantum state x) with the following complete vector representation as:
(o/=1 Ry Py 10)81=
[cos T1
coss D2
6 ....
cos I1
(x)-
(5)
sin 21
sin 22
sin 2z
Proof. Since each element 2; in the vector x can be written as (2:)) = cos (0) + sin 1, the quantum state (x) can be written as:
cos sin Φ
coo x2
( coos
(6)
sin T2
sin 11
When the vector goes through the quantum tensor network, which implies the following as: Ry (2x:)((0) = cosæ; (0) + sinz; [1) (as)..
The preceding equation, in turn, implies that Eq. (5)
Theorem I builds connection between the vector x and the quantum state (x). and the resulting
(x) is taken as the quantum embedding the inputs to VQC. Since @f_ Ry (2x;) unitary linear operator, there is no information loss incurred during the stage of quantum encoding. Furthermore, if the input is multiplied with constant T. we obtain the following term as:
R2x;); is a reversely
[oos [cos(TTa)) TTI))
cos (22()) sin (TI2)
coos( TT
(oL Ry (TTZ))
10)07_
sin T21)
...
sin(TZi
(8)
which corresponds to Figure (b).
5

---

Under review as a conference paper at ICLR 2022
10)
Rylag)
Ry(Bp)
RAy
10)
Rylag)
Ry(A))
RA(r.)
22
10)
Ry(ag)
Ry(P))
RA(s))
10)
Ry(a)
Ry(())
RAA
(e) Variational quantum circuit: the dashed square indicates repeated model with CNOT gates for entangling quantum states, and Ry(a), (f)) Ry(r) represent Paull-X, Y. Z gates with free parameters P.Y
sin /
0
cOs
sin
2
exp(-15)
R,(0)
Ry(a)
R
sin
Cos
0
0
-isn
COS
expri")
Pauli-Y
CNOT gate
Pauli-X
Pauli-Z
representation Quantum gates applied VQC
Figure 4: A framework of variational quantum circuit.
4.3 THE FRAMEWORK OF VARIATIONAL QUANTUM CIRCUIT
The framework of VQC is shown in Figure 4 (a), where 4 qubit wires are taken into account, and the CNOT gates aim mutually entangling the channels such that (21), (222). (.n)) and (ara) lie in the same entanglement state. The Pauli-X, Y, Z. gates Rx()) Ry(.) and Rz(1) with learnable parameters (a1, 31, 71). J. (a2, 32. 72). (at.. 33, 73). (ax4, 74) are built to set up the learnable part. Being similar to the unitary operators of Ry (a). (3) and Rz (1), which are defined in Figure 4 (b), are separately associated with the rotations along X-axis and Z-axis by the given angles of B and y. Besides, the quantum cireuits in the dash square be repeatedly copied to compose deeper architecture. The outputs of VQC are connected to the measurement which projects the quantum states into a certain quantum basis that becomes classical scalar 2;.
As for the end-to-end training paradigm for QTN-VQC, the learnable parameters come from the VQC and TTN models, and they should be updated by applying the back-propagation algorithm based on the Adam optimizer. Given D) qubits and И depths, there are totally 3DH trainable parameters for VQC. Consequently, there are Σ IE-IFa & + 3DH parameters for QTN- VQC. On the other hand, the Dense-VQC model possesses more model parameters than QTN-VQC TTk Pk-17% + 3DH vs. Ek Pk-17k + 3DH).
5 CHARACTERIZING REPRESENTATION POWER OF QTN-VQC
This section focuses on analyzing the representation power of QTN-VQC. As shown in Figure 5 given d qubits and a target quantum state (z) & &2 la)), since Ho is known a linear operator and T is defined as a definite mapping from input x to the unitary matrix Us. the representation power of QTN-VQC is determined by how TTN can approximate the classical vector (H lz:). To understand the expressiveness of TTN, we first start with the discussion on the expressive capability of Dense- VQC (a dense layer is taken for dimension reduction) and then generalize to QTN- Based on the universal approximation theorem (Cybenko, 1989 Barron 1994) for a feed-forward neural network, we derive the following theorem as:
Theorem 2. Given target vector (H with dense layer connecting to D qubits, then
lz)), there exists a feed-frwaard neural network faense
C
T (Hz' fdense (v) 111 S VD'
where the activation function tanh(-) ( is imposed upon the dense layer, and C. is a constant associ- ated with the target vector T (H, lz))

---

Under review as a conference paper at ICLR 2022
10:10; / (0) 10:10;10)
Oimension Reduction (TTN/ Dense)
(H
(k))
Tensor Product Encoder
Ho x)
Variational Quantum Cirouit
Hx
Hg
Tx(+)
Figure 5: An illustration of analyzing the representation power of QTN-VQC.
Since TTN is compact TT representation of dense layer, by modifying Theorem) 2 for TTN, we can also derive the upper bound on the approximation error as follows:
Theorem 3. Given a target vector Tx (Ho (z)), there exists TTN, denoted as fTTN, with a TT layer connecting to D qubits, then
C
1 (H) -
(10)
VD
where 1 D, the Sigmoid activation function is imposed upon the TTN model, K denotes the dilti-dimeesionall order, C is a constant associated with the target vector Tx TH, lz))
Comparing the upper bounds, is observed that TTN attain identical upper bound as the dense layer on the approximation error because IT D2 D. That implies that TTN can at least maintain the representation power of dense layer. Besides, the number of qubits D is key factor determining the upper bound on the approximation error. However, D a small fixed number on NISQ device, and larger number of qubits D is expected to further improve the representation power of QTN-VQC. However, the computational costs of classical simulation may grow exponentially with the increasing number of qubits, and small number of qubits have to be considered practice.
EXPERIMENTS AND RESULTS
6.1 EXPERIMENTAL SETUPS
We assess our QQC based end-to-end learning system on the standard MNIST. MNIST is a dataset for the task of 10 digit classification, where there are 50000 and 10000 28 x 28 image data assigned for training and testing, respectively. The full MNIST dataset challenging for quantum machine learning algorithms, and many works only consider 2-digit classification the MNIST task (Wang et al.] 2021] Chen et al.) 2020a). Moreover, the image data are separately reshaped into 784 dimensional input vectors. Dense-VQC and PCA-VQC
taken baselines to compare with our QTN-VQC model. Dense- VQC denotes that dense layer is used for dimension reduction, and PCA-VQC refers using principal component analysis (PCA) extract low-dimensional features before training the VQC parameters.
experimental
As for the experiments of QTN-VQC, the image data are reshaped into 3-order 7 16 x 7 tensors. We set small TT-ranks as {1. 2, 2. to reduce the computational cost of TTN. the image data are represented the TT format according Eq. (3) before going through the TTN model. Since qubits are used for the quantum encoding, the output of TTN needs to configure the tensor format as 2, which results in dimensional output vectors. Besides, the model parameters of QTN-VQC are randomly initialized based on the Gaussian distribution, and the prok-propagtttoon algorithm is applied to train the models. The Sigmoid function is utilized for the hidden layers of TTN.
To be consistent with VQC, the weight of the dense layer for Dense- VQC configured as the shape of 784 x 8. Although Dense- Dense-VQCis a hybrid lassical-quanteee model, the training process of Dense- VQC can also be set as an end-to-end pipeline and the weights the dense layer are updated during the training stage. The Sigmoid function used for the dense layer. On the other hand, PCA

---

Under review as a conference paper at ICLR 2022
employed to reduce the feature dimension 8, and the resulting low-dimensional features further encoded into quantum states. Consequently, PCA-VQC admits the VQC parameters solely to be updated during the training stage. A standard AlexNet (landola al. 2016) is employed to constitute an AlexNet-VQC to compare the performance.
Moreover, VQC layers constructed to form deep model, and the outputs of the VQC model are connected to 10 classes with a non- con-trainable matrix. The back- prock-propggation algorithm based on the Adam optimizer with a learning rate of 0.001 is employed for the model training. The loss of cross-entropy (CE) is utilized as the objective function during the training stage, and it is also taken as the metric to evaluate the model performance. We leverage the tools of Pennylane (Bergholm et al. (018)) and Py PyTorch (Paszke et al.) 2019) to simulate the model performance. In particular, we separately simulate the model performance with noiseless quantum circuits and noisy quantum circuits corrupted by quantum noises from IBM quantum machines.
6.2 EXPERIMENTAL RESULTS OF NOISELESS QUANTUM CIRCUIT
Table shows the final results of the models on the test dataset. QTN-VQC owns much fewer model parameters than Dense-VQC (328 vs. 6416) and attains even higher classification accuracy than Dense-VQC (91.43% vs. 88.54%) and lower loss values than Dense-VQC (0.3090 vs. 0.4132). However, PCA-VQC with 144 trainable VQC parameters attains the worst performance by all met- rics, which implies that a trainable quantum embedding is of significance to boost experimental performance. Although our empirical results cannot reach the state-of-the-art classification perfor- mance of classical ML algorithms, our empirical results demonstrate the advantages of QTN-VQC over the PCA-VQC and Dense-VQC counterparts. With the development of more powerful quan- tum devices supporting more qubits, the representation power of QTN-VQC can be improved and better experimental results could be attained. Moreover, AlexNet-VQC achieves better results than QTN-VQC 192881%vs.9 911.99.44 43%). but it involves more model parameters than QTN-VQC.
Table Empirical results on the MNIST test dataset under the noiseless quantum eircuit setting.
Models
Params
CE
Acc (%).
VQC
144
0.5877
82.48 1.02
Dense- VQC
6416
0.4132
88.54 = 0.73
AlexNet- VQC
3.25 x 106
0.2562
92.81 + 0.47
QOC-VQC
328
0.3090
91.43 0.51
6.3 EXPERIMENTAL RESULTS OF NOISY QUANTUM CIRCUIT
To empirically validate the effectiveness of our proposed VQC algorithm, we proceed with the simulation of the practical experiments with noisy quantum circuits. More specifically. we follow an established noisy circuit experiment with the NISQ device suggested by (Chen et al. (020b). One major advantage of the setups is observe the robustness and preserve the quantum advantages of a deployed VQC with physical settings being close to quantum processing unit (QPU) experiments without an executive queuing time. As for the detailed setup, we first use an IBM Q 20-qubit machine to collect channel noise in the real scenario for a deployed VQC and upload the machine noise into our Pennylane- lanns-Qskn-ii simulator (denoted as Acc,20- We provide depolarizing noisy circuit simulation (denoted as AcCdepo) based on a depolarizing channel attained from (Nielsen & Chuang 2010) with a noise level of 0.1. As shown in Table 2, the quantum noise brings about the performance degradation of all models, but our proposed QTN-VQC consistently outperforms PCA-VE PQC-VQC and Dense-VQQ VQC the condition of noisy quantum circuits. particular, QTN-VQC can even outperform the AlexNet- VQC counterpart in noisy circuit conditions.
6.4 FURTHER DISCUSSIONS
The above experimental results show the advantages of QTN-VQC over Dense VQC and PCA- VQC in the scenarios with noiseless and noisy quantum circuits. Next, will further discuss the representation power of QTN-VQC based on two factors: (1) the activation function used in TTN; (2) the number of qubits.
8

---

Under review as a conference paper at ICLR 2022
Table 2: Empirical results the MNIST test dataset under the noisy quantum circuit setting.
Models
Params
Acc,20 (%)
AcCo (%)
PCA-VQC
144
81.23
1.34
83.12 + 1.17
Dense- VQC
6416
84.55 + 1.22
86.09 + 1.04
AlexNet- VQC
3.25 x 10
87.46
1.34
87.86 + 1.08
QTN-VQC
328
88.12
1.09
89.32 1.07
6.4.1 THE ACTIVATION FUNCTION USED IN TTN
Table 3 compares the results of QTN-VQC based on different activation functions. Our simula- tion on noiseless quantum circuits shows that the non non-linear activation functions can bring more performance gain than linear one, but the Sigmoid function attains a better performance than the Tanh and ReLU counterparts in our experiments. Our experiments also correspond to the universal approximation theory for QTN-VQC in Theorem
Table 3: Comparing performance of QTN-VQC with and without activation function.
Models
CE
Acc (%).
QTN-VQC (Linear)
0.4958
86.16 = 0.65
VTN- (Tanh)
0.4792
87.12 + 0.51
QTN-VQC (ReLU)
0.3764
89.56 = 0.49
QTN-VQC (Sigmoid)
0.3090
91.43 = 0.54
6.4.2 THE NUMBER OF QUBITS
Finally, we investigate the effects of the number of qubits on the performance of QTN-VQC by increasing the qubits from 8 to 12 and 16. Accordingly, the output of TTN is configured tensor format of 2 x 3 x 2, and the model size is increased from 328 to 464 and 600, respectively. Our experiments show that the baseline performance of QTN-VQC can be further improved by increasing the number of qubits, which implies that more qubits are likely to possess higher accuracy.
Table Comparing performance of QTN-VQC with fewer qubits.
Models
Params
Acc (%)
QTN-VQC (8 qubits)
328
0.3090
91.43 + 0.51
QTN-VQC (12 qubits)
464
0.2679
92.36 + 0.62
QTN-VQC (16 qubits)
600
0.2355
92.98 + 0.52
7 CONCLUSIONS
This work proposes a genuine end-to-end learning framework for quantum neural networks based on QTN-VQC. QTN consists of TTN for dimension reduction and a TPE framework for generating quantum embedding. The TTN model is a compact representation of dense layer to classically simulate quantum machine learning algorithms. Our theorem on the representation of QTN-VQC shows that the number of qubits is inversely related to the approximation error of QTN-VQC and the non-linear actívation plays important role. Our experiments compare our proposed QTN-VQC with Res- EQC. Lense-VQC and PCA-VQC. Our simulated results demonstrate that QTN-VQC obtains better experimental performance than Dense-VOR and PCA-VQC with both noiseless and noisy quantum circuits, and achieves marginally worse performance than AlexNet-VQC. Besides, results justify our theorem the representation power of QQN-VQC.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Andrew R Barron. Approximation and Estimation Bounds for Artificial Neural Networks. Machine Learning, 14(1):115-133, 1994.
Marcello Benedetti, Erika Lloyd, Stefan Sack, and Mattia Fiorentini. Parameterized Quantum Cir- cuits as Machine Learning Models. Quantum Science and Technology, 4(4):043001, 2019.
Ville Bergholm, Josh Izaac, Maria Schuld, Christian Gogolin, M Sohaib Alam, Shahnawaz Ahmed, Juan Miguel Arrazola, Carsten Blank, Alain Delgado. Soran Jahangiri, et al. Pen- nylane: Automatic Differentiation of Hybrid Quantun-Classical Computations. arXiv preprint arXiv: 1.04968, 2018.
Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, and Seth Lloyd. Quantum Machine Learning. Nature, 549(7671)) 1): 2017.
Samuel Yen-Chi Chen, Chih-Min Huang, Chia-We Wei Hsing, and Ying-Jer Kao. Hybrid Quantum- Classical Classifier Based on Tensor Network and Variational Quantum Circuit, arXiv preprint arXiv:201 I. 14651, 2020a.
Samuel Yen-Chi Chen, Chao-Han Huck Yang. Jun Qi, Pin-Yu Chen, Xiaoli Ma, and Hsi-Sheng Goan. Variational Quantum Circuits for Deep Reinforcement Learning. IEEE Access, 8:141007- 141024, 2020b.
George Cybenko. Approximation by Superpositions of Sigmoidal Function. Mathematics of Controi, Signals and Systems, 2(4):303-314, 1989.
Li Deng, Jinyu Li, Jui-Ting Huang, Kaisheng Yao, Dong Yu, Frank Seide, Michael Seltzer, Geoff Zweig, Xiaodong He, Jason Williams, et al. Recent Advances in Deep Learning for Speech Research Microsoft. Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1604-8608. 2013.
Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, and Dacheng Tao. Expressive Power of Parametrized Quantum Circuits. Physical Review Research, 2(3):033125, 2020.
Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Dacheng Tao, and Nana Liu. Quantum Noise Protects Quantum Classifiers Against Adversaries. Physical Review Research, 3(2)0223153, 2021.
Vedran Dunjko. Inside Quantum Black Boxes. Nature Physics, pp. 1-2, 2021.
Vedran Dunjko and Hans Briegel. Machine Learning & Artificial Intelligence in the Quantum Domain: A Review of Recent Progress. Reports on Progress in Physics, 81(7):074001, 2018.
Vedran Dunjko, Jacob M Taylor, and Hans Briegel. Quantum- Machine Learning. Phys- ical Review Letters, (17(13):13050) 2016.
Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. Quantum Approximate Optimization Algo- rithm. arXiv preprint 1rXiv:1411.4000 2014.
David H Freedman. Hunting for New Drugs with AL. Nature, 576(778)):549-552. 2019.
Yangguang Fu, Mingyue Ding, and Chengping Zhou. Phase Angle-Encoded and Particle Swarm Optimization Applied to Dimeessnimssi Route Planning for UAV. 1EEE Transactions on Systems, Man, and Cybernetics-Pur A: Systems and Humans, 42(2):511-526, 2011.
Xun Gao, Zhengyu Zhang, and Luming Duan. An Efficient Quantum Algorithm for Generative Machine Learning. arXiv preprint arXiv: 1.00088, 2017.
Timur Garipov, Dmitry Podoprikhin, Alexander Novikov, and Dmitry Vetrov. Ultimate Tensoriza- tion: Compressing Convolutional and FC Layers Alike. arXiv preprint 16 1.032114 2016.
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep Learning, volume 1. MIT Press, 2016.
10

---

Under review as a conference paper at ICLR 2022
Maxwell Henderson, Samriddhi Shakya, Shashindra Pradhan, and Tristan Cook. Quanvolutional Neural Networks: Powering Image Recognition with Quantum Circuits. Quantum Machine In- telligence, 2(1):1-9, 2020.
Hsin-Yuan Huang, Michael Broughton, Masoud Mohseni, Ryan Babbush, Sergio Boixo, Hartmut Neven, and Jarrod R McClean. Power of Data in Quantum Machine Learning, Nature Communi- cations, 12(1):1-9, 2021.
Thomas Huckle, Konrad Waldherr, and Thomas SerbiigeHrrrrüiggen. Computations in Quantum Tensor Networks. Linear Algebra and its Applications, 438(2):750-781, 2013.
William Huggins, Piyush Patil, Bradley Mitchell, K Birgitta Whaley, and E Miles Stoudenmire. Towards Quantum Machine Learning with Tensor Networks. Quantum Science and Technology, 4(2):024001, 2019.
Forrest N landola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William Dally, and Kurt Keutzer. SqueezeNet: AlexNet-level Accuracy with 50x Fewer Parameters and; 0.5 MB Model Size. arXiv preprint I 2016.
Sofiene Jerbi, Casper Gyurik, Simon Marshall, Hans J Briegel, and Vedran Dunjko. Variational Quantum Policies for Reinforcement Learning. arXiv preprint arXiv:2103.05577 2021.
Xuanyu Jin, Jiajia Tang. Xianghao Kong, Yong Peng, Jianting Cao, Qibin Zhao, and Wanzeng Kong- CTNN: A Convolutional Tensor-Train Neural Network for Multi-Task Brainprint Recognition. Proc. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 29:103-112, 2020.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly Accurate Protein Structure Prediction with AlphaFold. Nature, 596(7873):583-58) 2021.
Abhinav Kandala, Antonio Mezzacapo, Kristan Temme, Maika Takita, Markus Brink, Jerry M Chow, and Jay M Gambetta. Hardware-Efficient Variational Quantum Figensolver for Small Molecules and Quantum Magnets. Nature, 549(7671):242-225) 2017.
Iordanis Kerenidis, Jonas Landman, and Anupam Prakash. Quantum Algorithms for Deep Convo- lutional Neural Networks. In Proc. International Conference on Learning Representations, 2020.
Frank Leymann and Johanna Barzen. The Bitter Truth About Gate-Based Quantum Algorithms In the NISQ Era. Quantum Science and Technology, 5(4):044007, 2020.
Jin-Guo Liu and Lei Wang. Differentiable Learning of Quantum Circuit Born Machines. Physical Review A, 9866):063324, 2018.
Seth Lloyd, Maria Schuld, Aroosa Ijaz, Josh Izaac, and Nathan Killoran. Quantum Embeddings for Machine Learning. arXiv preprint wxXiv2201.003622 2020.
Jarrod R McClean, Jonathan Romero, Ryan Babbush, and Alán Aspuru- Auzuk. Guzik. The Theory of Variational Hybrid Quantum -lasssicall Algorithms. New Journal of Physics. 18(2):02302,, 2016.
Jarrod McClean, Sergio Boixo, Vadim N Smelyanskiy, Ryan Babbush, and Hartmut Neven. Bar- ren Plateaus in Quantum Neural Network Training Landscapes. Nature Communications, 9(1):
1-6, 2018.
Kosuke Mitarai, Makoto Negoro, Masahiro Kitagawa, and Keisuke Fujii. Quantum Circuit Learning. Physical Review A, 98(3):032309, 2018.
Valentin Murg, Frank Verstracte, Örs Legeza, and Reinhard M Noack. Simulating Strongly Corre- lated Quantum Systems with Tree Tensor Networks. Physical Review B, 82(20):205105, 2010.
Michael A Nielsen and Isaac Chuang. Quantum Computation and Quantum Information. Cam- bridge, 2010.
Alexander Novikow, Dmitry Podoprikhin. Anton Osokin, and Dmitry Vetrov. Tensorizing Neural Networks. In Proc. Advances in Neural Information Processing Systems, 2015.

---

Under review as a conference paper at ICLR 2022
Román Orús. Tensor Networks for Complex Quantum Systems. 538-550, 2019.
Nature Reviews Physics, 1(9):
Ivan Oseledets. Tensor-Trai Decomposition. SIAM Journal on Scientific Computing, 33(5): 2295-2317, 2011.
Mateusz Ostaszewski, Lea M Trenkwalder, Wojciech Masarezyk, Eleanor Scerri, and Vedran Dun- jko. Reinforcement Learning for Optimization of Variational Quantum Circuit Architectures. arXiv preprint arXiv:2103. 2021.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An Imperative Style, Riihh-Performance Deep Learning Library. In Proc. Advances in Neural Information Processing Systems, volume 32, pp.
2019.
John Preskill. Quantum Computing in the NISQ Era and Beyond. Quantum, 2:79, August 2018. ISSN 2521-327X.
Jun Qi, Hu Hu, Yannan Wang, Chao-Han Huck Yang, Sabato Marco Siniscalchi, and Chin-Hou Hui Lee. Exploring deep hybrid tensor-to-vector network architectures for regression based speech enhancement. arXiv preprint arXiv:2007.13024 2020a.
Jun Qi, Hu Hu, Yannan Wang, Chao-Han Huck Yang, Sabato Marco Siniscalchi, and Chin-Hup Multi-Channel
Lee. Tensor-to- terstorto-Vector Vector Regression for Multi- Channel Speech Enhancement Based Tensor- Train Network. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 7504-7508, 2020b.
Valeria Saggio, Beate E Asenbeck, Arne Hamann, Teodor Strömberg, Peter Schiansky, Vedran Dun- jko, Nicolai Friis, Nicholas C Harris, Michael Hochberg, Dirk Englund, al. Quantum Speed-ups in Reinforcement Learning. In Quanium Nanophotonic Materials, Devices, and Systems 2021, volume 11806, pp. 11806N.. International Society for Optics and Photonics, 2021.
Maria Schuld and Nathan Killoran. Quantum Machine Learning in Feature Hilbert Spaces. Physical Review Letters, 124(4))040504, 2019.
Maria Schuld and Francesco Petruccione. Supervised Leurning with Quantum Computers, vol- ume 17. Springer, 2018.
Maria Schuld, Ilya Sinayskiy, and Francesco Petruccione. An Introduction to Quantum Machine Learning. Contemporary Physics, 57(2) 2015.
Pierre Sermanet, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, and Yann LeCun. Over- feat: Integrated Recognition, Localization and Detection Using Convolutional Networks. Proc. International Conference on Learning Representations, 2014.
Andrea Skolik, Sofiene Jerbi, and Vedran Dunjko. Quantum Agents in The Gym: A Variational Quantum Algorithm for Deep Q-Learning arXiv preprint 10Xiv:2103.1510 2021.
Eric Smalley. Al-Powered Drug Discovery Captures Pharma Interest. Nature Biotechnology. 35(7): 604-606, 2017.
Andrei N Soklakov and Rüdiger Schack. Efficient State Preparation for A Register of Quantum Bits. Physical review A, 73(1):01230, 2006.
Andros Tjandra, Sakriani Sakti, and Satoshi Nakamura. Compressing Recurrent Neural Network with Tensor- Train. In Proc. International Joint Conference on Neural Nenworks, pp. 1451-4458
2017.
Hanrui Wang, Yongshan Ding, Jiaqi Gu. Yujun Lin, David Z Pan, Frederic T Chong, and Song Han. Quantumnas: Noise-Adaptive Search for Robust Quantum Circuits. arXiv preprint arXiv:2107. 107.445. 2021.
12

---

Under review as a conference paper at ICLR 2022
Chao-Han Huck Yang. Jun Qi, Samuel Yen-Chi Chen, Yu Chen, Sabato Marco Siniscalchi, Xiaoli Ma, and Chin-Hui Lee. Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition. In Proc. IEEE International Conference Acoustics, Speech and Signal Processing, pp. 6523-6527, 2021.
Yinchong Yang, Denis Krompass, and Volker Tresp. Tensor-Train Recurrent Neural Networks for Video Classification. In Proc. International Conference on Machine Learning, pp. 3891-3900, 2017.
R Yu, S Zheng, A Anandkumar, and Y Yue. Long-term Forecasting Using Tensor-Train RNNs. arXiv preprint urXiv: 31, 2017.
13

---

Under review as a conference paper at ICLR 2022
A
APPENDIX
The section of appendix includes the proofs for Theorem 2 and Theorem
A.1 PROOF FOR THEOREM 2
Proof. Theorem 2 is derived from the modification of the universal approximation theory proposed by Barron (994); )
Cybenko (1989). The universal approximation theory shown in Lemma I which suggests that a feed-forward neural network with d neurons can approximate any continuous function with arbitrarily small c.
Lemma 1. Given continuous fancctifunctin f: E - IR, we can employ 2-layer neural network with non-linear activation f: RR. + E. such that
1 C
- fls ff- VD
(11)
where J denotes the number of neurons, and Cj is a constant associated with f. In particular, for r 2 1, Cy satisfies the following condition as:
l1 +
E
| fl SCj.
(12)
A.15 1(--l gr
where D& f
Vi V v*
To associate Lemma with our Theorem 2 the target function is replaced with the target vector H H Iz), then there is neural network with dense layer connected to D qubits such that
| H Iz) - fdense fd)nse(y)ll s VD C,
(13)
where related the target vector Hx' 1z).
A.2 PROOF FOR THEOREM 3
Proof. Assume that X' JTTN & Hxx (z) and the TT decomposition of target vector is {&i, Re,,, Rx}. then we obtain
H1²11)
K
| H (TTN(J))l = 1| X1I
II
& Xx
(14)
On the other hand, denote vec()i)) and vec(Xx) as the vectorization of the tensors Vd. and Xx. respectively. We also define I[ Ix 1, Wy E x as the TTN parameters, and also define We E xne-rri as the matricization We. Moreover, refers to non-linear activation function.
Since vec(Xt)) = oW vec()x)) that corresponds to a dense layer, we can obtain that
1 C.
11
Xulli
(15)
VDx
In sum, further obtain
1
| 2) frrw(y)l
< II
| Xalli - II
C,
(16)
DD
R=1
k=1
where [ D₃ D.
n
14

---

Under review as a conference paper at ICLR 2022
0.90
Dense VQC
QTN VOC
0.85
PCA_VQC
0.80
The CE Loss
0.75
0.70
0.65
Epoch
Figure comparison of convergence rates for different models.
B APPENDIX
This section includes additional experimental simulations. First, we assess the settings of TT-ranks, and then we compare the convergence rates QTN-VQU and Dense- Dense-VQC in the experiments.
B. EXPERIMENTS ON TT-RANKS FOR QTN-VQC
Table 5 corresponds to the experiments of QTN-VQC with 8 qubits and the Sigmoid function. The empirical results suggest that the larger TT-ranks cannot result in better results than the smaller ones. The main reason is that the TT-ranks can correspond to manifold, and there may potentially exist optimal manifold with smaller TT-ranks that corresponds to the best performance.
Table 5: Comparing performance of different TT-ranks for STN-VQC
TT-ranks
Params
CE
Acc (%)
2,
328
0.3090
91.43 0.51
11 4, 4,
1
768
0.3082
91.46 = 0.53
1, 6, 6,
1464
0.3079
91.47 + 0.52
B.2 A COMPARISON OF CONVERGENCE RATES
Next, analyze the computational complexity for TTN for QTN-VQC. In more detail, given the TT-ranks {r1, 72, TKh. multi-dimensional tensor W is factorized into several K-order tensors W2 E Rr X Xrs, the computational complexity of the feed-forward process is in the scale of O(K max; Is max; (maxx rk)** K.. In contrast, the computational overhead for dense layer is in the scale of O[ Is II. Jz). It means that smaller TT-ranks can reduce the computational cost for QTN-VQC, which explains that smaller TT-ranks 21, 2, is configured in our experiments of QTN-VQC.
Empirically, we compare the convergence rates of different models on the test data in our experi- ments. our experimental settings with the Tanh activation function and qubits, the QTN-VQC model consistently attains a faster convergence rate than the Dense-Von and PCA-VQC counter- parts. Moreover, Table 6 compares the absolute running time QTN-VQQ with Bense-VQQ and AlexNet- VQC. Since experiments are conducted the same GPUs and CPUs, the training time of all models can be comparable. Our evaluation shows that QTN- VQC marginally slower than Dense-VOR but it much faster than AlexNet-VOC
15

---

Under review as a conference paper at ICLR 2022
Table 6: Comparing performance of different TT-ranks for QTN-VQC
Models
Dense-VOS VQC
BexNee-VQC
QTN-VQC
Time/epochs (mins)
58
75
61
C EXPERIMENTS OF LABELED FACES IN THE WILD (LFW)
C.I EXPERIMENTAL SETUPS
The LFW is a dataset for the task of unconstrained face recognition, which is composed of 13000 images with the shape of [154, 154,3) 3]. The shape of We randomly split all the datasets into 11000 training data, 2000 test data. 16 qubits are used for VQC, and the shape of the input tensor is set as 22 147 22. The other settings are kept the same as the configurations for MNIST task.
C.2 EXPERIMENTAL RESULTS
Table presents the simulation results under the noiseless quantum circuit condition, while Table 7 demonstrates the empirical results in the setting of noisy quantum circuits. The QTN-VQC out- performs the Dense- VQC counterpart (92.15 vs. 91.27), and it owns much fewer model parameters (2816 vs. 1.1 10%). Although
The experimental results on the LFW dataset also highlight the advantages of QTN-VQC in terms of fewer model parameters and better empirical performance.
Table 7: Simulation results on the LFW test dataset under noiseless circuit condition.
Models
Params
CE
Ace (%)
Dense- VQC
1.1 x 10"
0.3011
91.27
0.25
AlexNet-VQC
2.3 x 107
0.2875
93.21 + 0.36
QTN-VQC
2816
0.2910
92.15 0.43
Table 8: Empirical results on the L.FW test dataset under the noisy quantum circuit setting.
Models
Params
Acc,20 (%)
AcCaepo (%)
Dense-VQC
1.1 x 106
88.65 + 1.22
87.23 + 1.04
AlexNet- VQC
2.3 x 10
89.76 1.34
88.66
1.08
QTN-VQC
2816
89.93 1.09
89.64
1.07
16


Reviews:
[{"review_id": "pEMp9sz5bLR", "reviewer": "Reviewer_JaBi", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n- The idea of combining QTT with VQC is interesting, which is helpful for quantum neural network. \n- The paper presents quantum neural network in a nicely understandable language for machine learning researchers although there are still some minor mistakes or confusion. \n\nWeakness:\n- The novelty of this paper is fair.  This paper combines QTT and VQC into one framework. TT has been applied to fully connected layer for model compression by many existing works.  VQC is well known quantum machine learning model.  TPE is a simple method to encode classical data as quantum state, which is also proposed by quantum inspired neural network (NeurIPS 2016).  \n\n- The experiment validation is done on MINIST dataset, however, the performance is far from the state-of-the-art performance by DNN, which cannot show how useful of quantum neural network.  Since quantum computing is based on simulation, can you increase the number of qubits and obtain better performance on this simply MINIST task? \n\n- It shows the number of parameters are much smaller by using QTT.  But how about computational cost and convergence rate when comparing with dense network?\n\n- Other experiments on larger datasets like CIFAR is expected, which is related to the scalability of the propose algorithm. \n\n- When TT is used to approximate the fully connected weights, the gradient exploding and diminishing often occur due to many of core tensors.  But the paper has not presented any algorithm related study and discussion.     \n\n- Input data x is reshaped into a k-order tensor, and then it is represented as TT format. However, this is the procedure of TT decomposition of data x, which is still challenging task and computational expensive. \n\n- In the Theorem 3, the upper bound is not related to TT-ranks, which seems not reasonable. Let us assume TTN with tt-ranks are all 1, can we achieve same upper bound as dense layer?\n\nMinor comments:\n- Eq. (2) is confusing, it should be that v is m-dimensional vector, and each v_i is 2-dimensional vector. \n- Typo in the line below eq.(4), the core tensor is X rather than W. \n- Eq. (6) and (9) are same. It is not necessary to be written twice. \n- The comparison of model parameters between QTN-VQC and Dense-TPE-VQC in the last line of Sec. 4 is also wrong. \n", "summary_of_the_paper": "This work proposes an end-to-end learning framework TTN-VQC for quantum neural networks. The main problem is that current quantum computer cannot handle large number of qubits.  The idea of this paper is applying QTT to perform dimensional reduction followed by quantum encoding TPE, which can be combined with existing quantum neural network (VQC) as an end-to-end learning model. ", "summary_of_the_review": "The paper is an improvement of quantum neural network algorithm.  The idea is simply but interesting for machine learning community.  However, novelty is fair, and the experimental evaluation is less convincing. Some important problems like computational cost, gradient exploding are not well discussed. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "b4sqUBqQ3hc", "reviewer": "Reviewer_nHEv", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The experiments show an improvement in performance from using TTN for dimensionality reduction, but these results would be stronger if another dataset besides MNIST was used, and if less trivial baselines were used.\n\n- There is very little new theoretical contributions. The tensor product encoding presented in theorem 1 has been extensively used before (since \\[Stoudenmire and Schwab 2016\\] at least), and theorems 2 and 3 are simple consequences of the universal approximation properties of dense shallow neural networks and tensor trains.\n\n- The use of TTN as a dimensionality reduction method is poorly justified in the paper, and not compared against sufficient baselines in the experiments. In particular, given that the dimensionality reduction is intended to be fully classical, it would make sense to use less trivial classical models as experimental baselines (e.g. a neural net with more than one layer).\n\n- The parameter counts given in Tables 1 and 2 are somewhat deceptive, as the stated requirement that input data is first converted into a TT format will involve a significant memory and runtime overhead. Converting data to TT format also seems unnecessary, and differs from the standard procedure for using matrices in TT format in place of dense matrices (e.g. \\[Novikov et al. 2015\\]).", "summary_of_the_paper": "A two-stage framework for quantum machine learning is presented, which uses a tensor train network (TTN) to reduce the dimensionality of input data, whose embedded form is fed to a variational quantum circuit (VQC) and measured to obtain an output. Several theoretical results about this framework are presented, and experimental results show the improved performance of this framework against a pair of similar baseline models.", "summary_of_the_review": "The paper's proposed method is a straightforward combination of several well-studied ideas, namely tensorization of linear weights, tensor product embeddings, and variational quantum circuits. The biggest strength of the paper is in its experimental results, but these consist of a few experiments on MNIST with a limited number of baseline models.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MVanzBjauWW", "reviewer": "Reviewer_8JcS", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Below, I describe the following issues of the paper:\n-\tThe main contribution is the idea of using TTN for compressing input data, which seems like a trivial approach. \n-\tMoreover, the selected TTN structure seems not to be optimal for image treatment. It is well known that, to obtain best compression rates of images with less distortion, data should be reshaped in order to collect small patches and group rows and columns indices. More specifically, given a IxJ image, we can obtain a good TT compression if we make the following transforms IxJ -> I1xI2x…xINxJ1xJ2x…xJN -> I1xJ1xI2xJ2x…INxJN -> (I1J1)x(I2J2)x…x(INJN).  I would suggest trying those optimal data transformations before applying the TT model which will contribute to obtain better classification accuracies.\n-\tOnly MNIST dataset was explored. It would be needed that other types of datasets were explored and analyzed.\n-\tI found the TT-ranks {1,2,2,1} too small. It would be usefult to have a visualization of the input data after compression to assess the introduced distortion. Also, it would be useful to report results with larger TT-ranks too.\n-\tProvided performance are poor compared to simple classical ML algorithms. For example, using a very simple Logistic Regression classifier can provide testing accuracy around 92%.\n-\tIn the definitions section, it should be clear that K is the number of input entries and d the number of qbits.\n-\tTheorem 1 is trivial and not needed. In fact, the proposed embedding of input data was already used in many papers, see for example [Stoudenmire, 2016].\nReferences:\n[Stoudenmire, 2016] Stoudenmire, E. M., & Schwab, D. J. (2016). Supervised Learning with Quantum-Inspired Tensor Networks. arXiv, 1605, arXiv:1605.05775.", "summary_of_the_paper": "As it was already studied in the literature, one way to implement a machine learning classifier in quantum computers is to convert classical input data like images into quantum states that are fed to a Variational Quantum Circuit (VQC). However, current quantum computers have a small number of qbits, which requires to perform a dimensionally reduction of the input datasets as a preprocessing step. In this paper, the authors propose to use data compression based on the Tensor Train Network (TTN) model, which is a very well-known compression technique. They compare TTN compression against a straightforward dense layer and the PCA-based dimension reduction techniques. The paper includes experimental results on MNIST dataset by simulating quantum circuits with up to 8qbits for noisy and noiseless scenarios.", "summary_of_the_review": "Strengths:\n- Quantum Machine Learning is a very important and growing field of research. This paper shows one potential way to use quantum computers for machine learning.\n\nWeaknesses:\n- The contribution of the paper is limited (see comment below). \n- Experimental results are also limited (see comments below)", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "M1CIhYIVu3V", "reviewer": "Reviewer_sUwU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The QML community has already explored many different ways for encoding classical input data into the parameterized angles in the quantum embedding circuit. For example, in [1], they have considered using ResNet for doing the encoding. It is not clear from this work that using the tensor-train network is beneficial compared to the plethora of existing methods (such as ResNet). The improvement over using a dense neural network is also marginal (+3% to 4% on MNIST).\n\nFurthermore, we can see that the prediction performance on MNIST is significantly worse than the state-of-the-art results (> 99% accuracy). A straightforward QML model is to consider a state-of-the-art CNN for MNIST and encode the output of the CNN into a single-qubit quantum circuit. We consider the single-qubit quantum circuit to act as an identity (so the quantum circuit is not doing anything). The performance of such a QML model will also achieve >99% accuracy because this trivial QML model is equivalent to the state-of-the-art CNN.\n\nTogether I think the main claim that \"Our experiments on the MNIST dataset demonstrate the advantages of QTN for quantum embedding over other quantum embedding approaches.\" is unjustified.\n\n[1] Lloyd, Seth, et al. \"Quantum embeddings for machine learning.\" arXiv preprint arXiv:2001.036", "summary_of_the_paper": "Quantum neural networks have two parts: a quantum embedding circuit that takes in classical data and embeds it into a quantum state, and a variational quantum circuit that learns a quantum circuit for evolving the quantum state before measurement.\n\nThe authors propose to encode the classical input data into the parameterized angle in the quantum embedding circuit using a tensor-train network (instead of a dense neural network). This results in a 3-4% improvement in prediction accuracy for MNIST (87.12% accuracy).", "summary_of_the_review": "It is not clear that the proposed method is better than the existing QML models. Hence, the main claim of this work is not justified.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "8NL3plJPdO", "reviewer": "Reviewer_Awxc", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\n1. The paper tackles a real limitation for QNN by replacing the dense NN with TTN and enabling end-to-end training.\n2. The author provides codes for reproducing, which is great.\n\nCons:\n\n1. The related work is insufficient. Since the TTN is the key proposal of this paper, it should be explained more in the related works. For example, as an existence module, how it works in the convolutional neural network and recurrent neural networks. What is the difference between this paper and previous literature, excluding the application difference?\n2. The ablation is highly insufficient. Especially, in table 3/4, only 1 non-linear activation and two # qubits options are evaluated.\n3. The technical contribution seems limited. Since the authors do not touch the QNN and only use an existing module TTN to replace the previous dense layer for pre-processing high-dimensional input.", "summary_of_the_paper": "Summary:\n\nThis paper is dedicated to designing an end-to-end learning framework for quantum neural networks. The authors design a novel quantum tensor network for dimension reduction and quantum embeddings generation, since it is quite related to the practical usage in real-world applications where only a small number of qubits could be supported on available NISQ computers at this moment. The key contribution of this paper is to leverage a tensor train network (TTN) to replace the dense layer for dimension reduction, which enables an end-to-end training process fully conducted in a quantum computer.", "summary_of_the_review": "Due to the limited technical contribution/novelty and insufficient experiments, I tend to borderline reject the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This submission proposes a trainable quantum tensor network (QTN) for quantum embedding generation on a variational quantum circuit (VQC), which is followed by a significant empirical study on the QTN-VQC performance on the MNIST test dataset. However, the discussion during the review period raises some serious concerns about both the theoretical and empirical contributions of the submission. 

On the theory side, we acknowledge the authors propose the use of tensor-train-network (TTN) as the dimension reduction layer of input features, design the tensor-product-encoding (TPE), and characterize the representation power of QTN-VQC.  However, a general feeling among all reviewers is that these contributions are kind of observational, rather than deep theoretical insights, based on existing results. For example, TTN is a well-established tool in dimension reduction, and the representation power of QTN-VQC can be derived as simple corollaries of the existing universal approximation theorem for a feed-forward neural network.  Moreover, the authors emphasize a lot that the use of TTN allows a genuine end-to-end quantum implementation of QTN-VQC because TTN can be relatively easier to implement as quantum circuits. However, the benefit of this end-to-end quantum implementation is rather unclear, especially for NISQ applications. In particular, because NISQ machines have limited quantum resources, should not a classical-quantum hybrid implementation be preferred in the interest of saving quantum resources? As genuine quantum implementation becomes the major motivation for using TTN, there is no other theoretical justification for selecting TTN. 

We appreciate the authors’ efforts in carrying out the empirical study and the active interaction during the discussion period. Unfortunately,  most of the reviewers are not convinced by the existing experimental results. This is partially because the authors seek to support a very strong claim that QTN-VQC would outperform the state-of-the-art classical solutions, whereas the limitation of current quantum devices prevents any empirical study of QTN-VQC at the scale comparable to the classical solutions. Moreover, as pointed out by one of the reviewers, there is hardly any existing evidence that quantum neural networks would be useful at all for commonly used datasets in NLP and vision. Given that, it is very unlikely that a conclusion could be reached by an empirical study on small-scale instances. Nevertheless, since the authors aim to compare QTN with other quantum embedding methods,  other kinds of experiments are possible without directly comparing with the state-of-the-art classical solutions. 

We believe that the submission would benefit a lot from addressing the concerns for both the theory and empirical parts and hope the authors would pursue it.
Decision: Reject