Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: A two-stage framework for quantum machine learning is presented, which uses a tensor train network (TTN) to reduce the dimensionality of input data, whose embedded form is fed to a variational quantum circuit (VQC) and measured to obtain an output. Several theoretical results about this framework are presented, and experimental results show the improved performance of this framework against a pair of similar baseline models.
main_review: - The experiments show an improvement in performance from using TTN for dimensionality reduction, but these results would be stronger if another dataset besides MNIST was used, and if less trivial baselines were used.

- There is very little new theoretical contributions. The tensor product encoding presented in theorem 1 has been extensively used before (since \[Stoudenmire and Schwab 2016\] at least), and theorems 2 and 3 are simple consequences of the universal approximation properties of dense shallow neural networks and tensor trains.

- The use of TTN as a dimensionality reduction method is poorly justified in the paper, and not compared against sufficient baselines in the experiments. In particular, given that the dimensionality reduction is intended to be fully classical, it would make sense to use less trivial classical models as experimental baselines (e.g. a neural net with more than one layer).

- The parameter counts given in Tables 1 and 2 are somewhat deceptive, as the stated requirement that input data is first converted into a TT format will involve a significant memory and runtime overhead. Converting data to TT format also seems unnecessary, and differs from the standard procedure for using matrices in TT format in place of dense matrices (e.g. \[Novikov et al. 2015\]).
summary_of_the_review: The paper's proposed method is a straightforward combination of several well-studied ideas, namely tensorization of linear weights, tensor product embeddings, and variational quantum circuits. The biggest strength of the paper is in its experimental results, but these consist of a few experiments on MNIST with a limited number of baseline models.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Quantum neural networks have two parts: a quantum embedding circuit that takes in classical data and embeds it into a quantum state, and a variational quantum circuit that learns a quantum circuit for evolving the quantum state before measurement.

The authors propose to encode the classical input data into the parameterized angle in the quantum embedding circuit using a tensor-train network (instead of a dense neural network). This results in a 3-4% improvement in prediction accuracy for MNIST (87.12% accuracy).
main_review: The QML community has already explored many different ways for encoding classical input data into the parameterized angles in the quantum embedding circuit. For example, in [1], they have considered using ResNet for doing the encoding. It is not clear from this work that using the tensor-train network is beneficial compared to the plethora of existing methods (such as ResNet). The improvement over using a dense neural network is also marginal (+3% to 4% on MNIST).

Furthermore, we can see that the prediction performance on MNIST is significantly worse than the state-of-the-art results (> 99% accuracy). A straightforward QML model is to consider a state-of-the-art CNN for MNIST and encode the output of the CNN into a single-qubit quantum circuit. We consider the single-qubit quantum circuit to act as an identity (so the quantum circuit is not doing anything). The performance of such a QML model will also achieve >99% accuracy because this trivial QML model is equivalent to the state-of-the-art CNN.

Together I think the main claim that "Our experiments on the MNIST dataset demonstrate the advantages of QTN for quantum embedding over other quantum embedding approaches." is unjustified.

[1] Lloyd, Seth, et al. "Quantum embeddings for machine learning." arXiv preprint arXiv:2001.036
summary_of_the_review: It is not clear that the proposed method is better than the existing QML models. Hence, the main claim of this work is not justified.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work proposes an end-to-end learning framework TTN-VQC for quantum neural networks. The main problem is that current quantum computer cannot handle large number of qubits.  The idea of this paper is applying QTT to perform dimensional reduction followed by quantum encoding TPE, which can be combined with existing quantum neural network (VQC) as an end-to-end learning model. 
main_review: Strength:
- The idea of combining QTT with VQC is interesting, which is helpful for quantum neural network. 
- The paper presents quantum neural network in a nicely understandable language for machine learning researchers although there are still some minor mistakes or confusion. 

Weakness:
- The novelty of this paper is fair.  This paper combines QTT and VQC into one framework. TT has been applied to fully connected layer for model compression by many existing works.  VQC is well known quantum machine learning model.  TPE is a simple method to encode classical data as quantum state, which is also proposed by quantum inspired neural network (NeurIPS 2016).  

- The experiment validation is done on MINIST dataset, however, the performance is far from the state-of-the-art performance by DNN, which cannot show how useful of quantum neural network.  Since quantum computing is based on simulation, can you increase the number of qubits and obtain better performance on this simply MINIST task? 

- It shows the number of parameters are much smaller by using QTT.  But how about computational cost and convergence rate when comparing with dense network?

- Other experiments on larger datasets like CIFAR is expected, which is related to the scalability of the propose algorithm. 

- When TT is used to approximate the fully connected weights, the gradient exploding and diminishing often occur due to many of core tensors.  But the paper has not presented any algorithm related study and discussion.     

- Input data x is reshaped into a k-order tensor, and then it is represented as TT format. However, this is the procedure of TT decomposition of data x, which is still challenging task and computational expensive. 

- In the Theorem 3, the upper bound is not related to TT-ranks, which seems not reasonable. Let us assume TTN with tt-ranks are all 1, can we achieve same upper bound as dense layer?

Minor comments:
- Eq. (2) is confusing, it should be that v is m-dimensional vector, and each v_i is 2-dimensional vector. 
- Typo in the line below eq.(4), the core tensor is X rather than W. 
- Eq. (6) and (9) are same. It is not necessary to be written twice. 
- The comparison of model parameters between QTN-VQC and Dense-TPE-VQC in the last line of Sec. 4 is also wrong. 

summary_of_the_review: The paper is an improvement of quantum neural network algorithm.  The idea is simply but interesting for machine learning community.  However, novelty is fair, and the experimental evaluation is less convincing. Some important problems like computational cost, gradient exploding are not well discussed.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: As it was already studied in the literature, one way to implement a machine learning classifier in quantum computers is to convert classical input data like images into quantum states that are fed to a Variational Quantum Circuit (VQC). However, current quantum computers have a small number of qbits, which requires to perform a dimensionally reduction of the input datasets as a preprocessing step. In this paper, the authors propose to use data compression based on the Tensor Train Network (TTN) model, which is a very well-known compression technique. They compare TTN compression against a straightforward dense layer and the PCA-based dimension reduction techniques. The paper includes experimental results on MNIST dataset by simulating quantum circuits with up to 8qbits for noisy and noiseless scenarios.
main_review: Below, I describe the following issues of the paper:
-	The main contribution is the idea of using TTN for compressing input data, which seems like a trivial approach. 
-	Moreover, the selected TTN structure seems not to be optimal for image treatment. It is well known that, to obtain best compression rates of images with less distortion, data should be reshaped in order to collect small patches and group rows and columns indices. More specifically, given a IxJ image, we can obtain a good TT compression if we make the following transforms IxJ -> I1xI2x…xINxJ1xJ2x…xJN -> I1xJ1xI2xJ2x…INxJN -> (I1J1)x(I2J2)x…x(INJN).  I would suggest trying those optimal data transformations before applying the TT model which will contribute to obtain better classification accuracies.
-	Only MNIST dataset was explored. It would be needed that other types of datasets were explored and analyzed.
-	I found the TT-ranks {1,2,2,1} too small. It would be usefult to have a visualization of the input data after compression to assess the introduced distortion. Also, it would be useful to report results with larger TT-ranks too.
-	Provided performance are poor compared to simple classical ML algorithms. For example, using a very simple Logistic Regression classifier can provide testing accuracy around 92%.
-	In the definitions section, it should be clear that K is the number of input entries and d the number of qbits.
-	Theorem 1 is trivial and not needed. In fact, the proposed embedding of input data was already used in many papers, see for example [Stoudenmire, 2016].
References:
[Stoudenmire, 2016] Stoudenmire, E. M., & Schwab, D. J. (2016). Supervised Learning with Quantum-Inspired Tensor Networks. arXiv, 1605, arXiv:1605.05775.
summary_of_the_review: Strengths:
- Quantum Machine Learning is a very important and growing field of research. This paper shows one potential way to use quantum computers for machine learning.

Weaknesses:
- The contribution of the paper is limited (see comment below). 
- Experimental results are also limited (see comments below)
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Summary:

This paper is dedicated to designing an end-to-end learning framework for quantum neural networks. The authors design a novel quantum tensor network for dimension reduction and quantum embeddings generation, since it is quite related to the practical usage in real-world applications where only a small number of qubits could be supported on available NISQ computers at this moment. The key contribution of this paper is to leverage a tensor train network (TTN) to replace the dense layer for dimension reduction, which enables an end-to-end training process fully conducted in a quantum computer.
main_review: Pros:

1. The paper tackles a real limitation for QNN by replacing the dense NN with TTN and enabling end-to-end training.
2. The author provides codes for reproducing, which is great.

Cons:

1. The related work is insufficient. Since the TTN is the key proposal of this paper, it should be explained more in the related works. For example, as an existence module, how it works in the convolutional neural network and recurrent neural networks. What is the difference between this paper and previous literature, excluding the application difference?
2. The ablation is highly insufficient. Especially, in table 3/4, only 1 non-linear activation and two # qubits options are evaluated.
3. The technical contribution seems limited. Since the authors do not touch the QNN and only use an existing module TTN to replace the previous dense layer for pre-processing high-dimensional input.
summary_of_the_review: Due to the limited technical contribution/novelty and insufficient experiments, I tend to borderline reject the paper.
review_title: null

Meta-review: This submission proposes a trainable quantum tensor network (QTN) for quantum embedding generation on a variational quantum circuit (VQC), which is followed by a significant empirical study on the QTN-VQC performance on the MNIST test dataset. However, the discussion during the review period raises some serious concerns about both the theoretical and empirical contributions of the submission. 

On the theory side, we acknowledge the authors propose the use of tensor-train-network (TTN) as the dimension reduction layer of input features, design the tensor-product-encoding (TPE), and characterize the representation power of QTN-VQC.  However, a general feeling among all reviewers is that these contributions are kind of observational, rather than deep theoretical insights, based on existing results. For example, TTN is a well-established tool in dimension reduction, and the representation power of QTN-VQC can be derived as simple corollaries of the existing universal approximation theorem for a feed-forward neural network.  Moreover, the authors emphasize a lot that the use of TTN allows a genuine end-to-end quantum implementation of QTN-VQC because TTN can be relatively easier to implement as quantum circuits. However, the benefit of this end-to-end quantum implementation is rather unclear, especially for NISQ applications. In particular, because NISQ machines have limited quantum resources, should not a classical-quantum hybrid implementation be preferred in the interest of saving quantum resources? As genuine quantum implementation becomes the major motivation for using TTN, there is no other theoretical justification for selecting TTN. 

We appreciate the authors’ efforts in carrying out the empirical study and the active interaction during the discussion period. Unfortunately,  most of the reviewers are not convinced by the existing experimental results. This is partially because the authors seek to support a very strong claim that QTN-VQC would outperform the state-of-the-art classical solutions, whereas the limitation of current quantum devices prevents any empirical study of QTN-VQC at the scale comparable to the classical solutions. Moreover, as pointed out by one of the reviewers, there is hardly any existing evidence that quantum neural networks would be useful at all for commonly used datasets in NLP and vision. Given that, it is very unlikely that a conclusion could be reached by an empirical study on small-scale instances. Nevertheless, since the authors aim to compare QTN with other quantum embedding methods,  other kinds of experiments are possible without directly comparing with the state-of-the-art classical solutions. 

We believe that the submission would benefit a lot from addressing the concerns for both the theory and empirical parts and hope the authors would pursue it.
Decision: Reject