Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: 
This paper uses the mathematical tools of quantum mechanics to realize a fast and accurate text classification algorithm. Specifically, in this paper, a document is regarded as the superposition state of words, and the classification is viewed as the document to collapse to another superposition state associated with the target class. The collapse probability of the document can be computed by the Born rule. Most of the experimental results are better than baseline models. In addition, the authors also give the interpretability analysis of the two methods. The interesting experimental results show that the classification algorithm is effective and interpretable.

strengths_and_weaknesses: From the overall perspective, the idea of this paper is interesting, and the algorithm proposed is small and exquisite. The experimental results show that the training time is greatly shortened, and the experimental accuracy is improved compared with baseline models.

However, the paper also has some weaknesses that need to be enhanced. First, the quantum-inspired models applied to supervised classification tasks is not the author's initiative. Moreover, in the experimental analysis, there is a lack of comparative experiments between this model and other quantum-inspired models. Second, the author uses quantum theory to analogize the relationship between documents and words, and the relationship between words and categories. This approach is based on intuition and has no strong theoretical basis. From the perspective of quantum measurement, this method is not reasonable. Third, the regularization lacks detailed analysis, and the use of entropy is incomprehensible. Fourth, some important information is not detailed in this paper, such as the necessity of using quantum theory, the method of quantum theory modeling prior knowledge, and what information the prior knowledge represents in the task. These makes people doubt whether it is necessary to introduce quantum theory into the problem raised in this paper. 

questions: Q1: The motivation of this paper says the prior knowledge results in more efficient algorithms. What is prior knowledge in this work, and does it give an intuitive understanding at this point?

Q2: The author says this work is the first supervised classification algorithm based on quantum theory. In my opinion, this view is not rigorous. Because many quantum-inspired models have been used to supervised classification tasks, such as QMNN[1], Complex-order[2]. In addition, there are some other quantum-inspired language approaches.e.g., quantum language models, quantum many-body wave language models. What is the advantage of this work over these language approaches?

Q3: On line 68, $|j\rangle$ is represented as word, and the document is represented as a superposition of state $|\psi\rangle = \sum_j \psi_j |j\rangle$. In quantum theory, the superposition state indicates that a quantum state is superimposed by its eigenstates, and will collapse to an eigenstate after measurement. However, the superposition state represent a document, which means that a word will be used to represent the document after measurement. Even if there is no measurement and the document is expressed as the weighted sum of word vectors in the code, its physical meaning is still unreasonable. 

Q4: On line 71, author represents each class k with a wave function $|\phi^{(k)}\rangle = \sum_j \phi_j^{(k)} |j\rangle$. Why the class k is the superposition state of word? Does this mean that class k will be represented by a word in the document after measurement?

Q5: In equation 5, the $p_{j|k}$ means the conditional probability of j given k. It is necessary to specify the relationship between j and k, and the reason why the conditional probabilities are used as the superposition state coefficients.

Q6: On line 80, the $x^{(n)}$ is feature vector. Is $x^{(n)}$ a word or $|j\rangle$? What is the relationship between $y^{(n)}$ and $y_k^{(n)}$? And the $y_k^{(n)}$ has no definition.


Q7: On line 85, author says “the j-th addend does not alter the ranking of the probabilities $u_k$, thus being irrelevant for the final classification”. However, each $\sqrt{p_{j|k} x_j}$ contributes to $u_k$ in the equation 6. Why the j-th addend is irrelevant for the final classification? The reason why entropy is used as weight is not given. Can you introduce the rationality of entropy? In the equation 10, is $u_k$ a normalized vector?

Q4: On line 136, $\Phi$ is a matrix of weights， and $\Phi_{ks}$ is a scalar. Why are they equal? Is that ambiguous? Similarly, $v=v_s$ is also confusing.

Q6: Whether it can be understood in this way, the 145 to 152 line gives the initialization process of BL, and the 153 to 155 line gives the initialization process of BL by BC.

Q7: On line 159, Why does $\rho e^{i\\theta} \overline{\varphi}_s^{(k)}$ have to be derived in this form? Can't $\overline{\varphi}_s^{(k)}$ be directly interpreted?

Q8: The novelty of this work is mainly Born rule, but most quantum-inspired models use quantum measurement theory to classify or judge. What is the essential difference between this work and other quantum-inspired models?

Q9: The reasons for choosing these baseline models need to be explained. Is there comparability between this work and other baseline models in ideas, methods and other aspects? Why 

[1] Li Q, Gkoumas D, Sordoni A, et al. Quantum-inspired neural network for conversational emotion recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2021, 35(15): 13270-13278.

[2] Wang B, Zhao D, Lioma C, et al. Encoding word order in complex embeddings[J]. arXiv preprint arXiv:1912.12333, 2019.

limitations: The authors conclude that there are limitations that they cannot explain why quantum-inspired algorithms are better than classical algorithms. In addition, I think the limitation of this work is also the proposal of the regularization term, and whether the regularization term can be repurposed and better explained by quantum theory.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a novel algorithm derived from Born's Rule in quantum physics for text classification. The paper discusses various aspects of this classification algorithm, including training, regularization, generalization, explainability, and computational complexity. A neural architecture called Born Layer is also developed. The empirical results show the superiority of the proposed algorithm in prediction accuracy and computational complexity over several classical baseline ML algorithms. This work is promising to be the first step to develop a brand new general classification algorithm with certain explainability inspired by quantum physics.
strengths_and_weaknesses: **Strength**

- I love the idea of a classification algorithm based on the theory of quantum physics. The authors have a comprehensive discussion of various aspects of the proposed classification algorithm.
- The algorithm is implemented with well-structured code, which should make the community easy to extend and try on more tasks.
- The algorithm shows impressive superiority over baseline algorithms in the prediction accuracy and training/inference efficiency.

**Weakness**

- The empirical result is limited. The major experiments are only conducted on 20NewsGroup. Also, the compared baselines are relatively weak. (But considering this is a preliminary work in this direction, this is acceptable.)

questions: - In Figure 2, the training time of KNN increases as the dataset size grows. This implies that the compared KNN is not a brute-force one. Moreover, I would expect to see more details to be elaborated on for traditional algorithms in the supplemental material.
- The Born Classifier shows a significant superiority over other classical algorithms when the dataset size is limited. This is surprising. Do you have any interpretation of this observation? Do you have such justification for other datasets?

limitations: The author does not discuss the limitations.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a novel text classification method (Born layer) based on quantum mechanics. The Born layer takes the input vector as the wave function and predicts the class of the given input with Born’s rule. The Born layer can be applied to any other neural network or function that transforms the input text into a feature vector. This paper uses the simple text transformation method, tokenization, for the transformation function, with the square root function derived by the Born rule. Experiments on three text classification datasets (20Newsgroup, R8, and R52) show the efficacy of the proposed method. The proposed method is also computationally efficient.
strengths_and_weaknesses: Strengths

1. The proposed method is novel.
2. This paper is the first trial that applies quantum theory to the text classification problem.
3. Experimental results support the validity of the proposed method.

Weaknesses

1. This paper is hard to follow. See the questions section for more details.
2. This paper does not provide any detailed description of their assumption that “text is a superposition of words.” What is the motivation that the authors see a text snippet as a superposition of words? How is this assumption brings a positive impact on text classification? What problem do existing studies have, and how does the proposed method resolve it?
questions: 1. Line 67: Is x_j unnormalized probability? Eq 4 states that x_j is a probability.
2. The Born layer is one of the main contributions of this paper. Do authors try any ablation study on the Born layer? e.g., apply the Born layer to other approaches and see the performance improvement.

Comments

1. In line 79, this paper describes P as the probability, but in line 83, the authors describe P as a vector. Please provide a more clear description.
2. Please provide a more description for the generalization form. What do the authors want to claim from the ablation study for the hyperparameters a and b?
3. The proposed method is applicable to other text classification tasks such as sentiment classification. More experiments on other popular text classification problems are recommended.
limitations: The authors have addressed the limitations. No negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper describes an interesting application of Born Rule to one of the most fundamental tasks in machine learning which is text classification. The Born Rule is a concept in quantum physics on estimating the probability of an object (or document in the context of this study) to collapse in one of the probable outcomes (category/class of the text). In this line, the paper frames text classification as a superposition of words where Born Rule can be integrated as a standalone classifier or with integration to a neural classifier. The authors show that the method outperform both traditional and neural network based methods on three datasets (NG, R8, and R52) by a considerable amount of margin especially with >1 epochs. There is also an emphasis on the efficient runtime obtained by using the proposed method with Born’s rule. 
strengths_and_weaknesses: 1. The paper is fairly easy to read, and follow, and contains an arguably sufficient level of technicality that would make even intermediate readers be interested in the concepts in quantum physics such as the Born rule. I also appreciate that the authors conveniently built the method on top of Scikit and Pytorch for ease of replication on the side of the readers. 
2. For this new approach, I expected a very definitive discussion on the limitations to describe the pitfalls of the proposed method in various applications and data types. Will it work on cases such as a sentiment classification task where some level of context is needed and there are cases of double negatives? Will it work on extremely noisy texts such as those collected from social media (Twitter)? What is the effect if we reduce the size of each text instance (say sentence classification instead of document-based classification)? 
3. Following #2, a definitive synthesis comparing TC with Born’s rule to existing methods of text classification (using language models, traditional method) should be in order. This can be tabulated for ease of reference.

With all the strengths and weaknesses of the paper, I believe it would still be beneficial for the ML community to learn from the paper provided the authors studiously highlight the limitations of the method. Thus, I would recommend acceptance on my part.

questions: 1. What was the motivation for traditional algorithms when the use of language models such as BERT are already considered baseline in text classification as they are equally accessible compared to the tested algorithms? Also, what were the criteria for choosing the three datasets when IMDB, Amazon, Yelp, AG are often used for the evaluation of newly-proposed TC methods (see Patel et al, NAACL 2021).
2. Are there any cases where TC with Born’s rule favors particular classes based on their confusion matrices? The paper only shows Accuracy scores so there is uncertainty if it resolves class imbalance or not. This can be a limitation of the method.

limitations: The paper covers a new method for implementing text classification that seems to outperform both traditional and some neural network-based methods in terms of speed and at par with accuracy scores. The authors are advised to exhaustively list what the proposed method is (not) capable of doing covering factors such as context, how texts is processed, type of task. Again, this can be described through a table.
review_title: null

Meta-review: metareview: This  paper has 2 accepts (7) and  2 borderline accepts (5). The average is 6.
The  modification of Reviewer Sqxh does not show in the system, but he stated as follows in our discussion “I tend to modify the score of this paper to five.”


This paper shows an algorithm that delivers outstanding text classification performance despite its extreme simplicity and speed. The quantum “explanation” for the algorithm is weak. The experiments were limited to somewhat simple text classification problems, but the authors added some convincing results on sentiment analysis in their rebuttal. The paper is clearly written and the provided code is well documented, so it should be reproducible (though none of the reviewers did rerun the experiments).  The authors provided a revised version that clarified some aspects (though it did not include the additional experiments provided in the rebuttal).

The consensus after discussion is accept, given that this paper may open a new class of simple and high performance classification algorithms. The explanation for their effectiveness remains an open problem, and this paper could be improved by opening the discussion.
We recommend the following modifications in order to mitigate overreaching claims that could make the authors look naïve:

- Quantum interpretation of a sentence as superposition of words. This seems to imply that detecting a single word is sufficient to classify a sentence, and that the best solution is a combination of these weak classifiers based on a single word or N-gram. The success of Adaboost on text classification testifies of the power of these methods. However such methods do not work so well when sentences are longer, or when there is a stronger compositionality, for instance sentiment analysis (IMDB, SST). The results on EmoInt provided in the refutation are a step in the right direction. They show the method can handle some level of context, thought sentences are still short. How could this method handle long sentences? The paper would benefit from a discussion about what the superposition of words hypothesis  implies, what are it limitations, and maybe how to overcome them.

- Table 1 provides preliminary results but some of the claims can be turn off for readers with experience in linear classifiers (SVM and LR) applied to text recognition, as the authors apparently failed to select the correct algorithm (SGD classifier). Comparisons with SVMs and LR in table1 suggest BC is 1 million times faster than SVMs. However, the same difference in speed can be observed within different Sklearn SVM implementations, depending on the algorithm. The SGD classifier, which supports both SVM and LR, has a computational complexity which is even better than the O(NJK) reported in the paper, as it depends on the number of non-zero features rather than the number of features J.
Furthermore, the accuracy provided for the SVM (79.4) also seems far below the SOTA. For instance, the reference [17] reports an accuracy of 82.27 on 20NG using TF-IDF SVMs (the most vanilla setting).
	
- As shown in Eq.(6), the model is fundamentally linear. Explainability by taking the feature with the highest weight is as ancient as ML itself. How this method performs better than traditional linear approaches should be better illustrated. The authors just say “The words appear semantically correlated to their respective class and do not contain neither noisy words, such as stopwords or punctuation, or words whose meaning is too general to be   representative of a specific class. "

- Novelty of this paper in the field of quantum-inspired classification. Even in the revised version, while they quote work on language modeling, the authors do not seem to have quoted any work on classification, even though it was pointed by the first reviewer (https://ojs.aaai.org/index.php/AAAI/article/view/17567). As pointed in their rebuttal, this is very different from their work, but they should mention that work.


confidence: Less certain
Decision: Accept