TextTN: Probabilistic Encoding of Language on Tensor Network
Authors: Peng Zhang, Jing Zhang, Xindian Ma, Siwei Rao, Guangjian Tian, Jun Wang
Keywords: Tensor Network, Language Representation, Natural Language Processing, Quantum Machine Learning, Entanglement Entropy
Abstract: As a novel model that bridges machine learning and quantum theory, tensor network (TN) has recently gained increasing attention and successful applications for processing natural images. However, for natural languages, it is unclear how to design a probabilistic encoding  architecture to efficiently and accurately learn and classify texts based on TN. This paper proposes a general two-step scheme of text classification based on Tensor Network, which is named as TextTN. TextTN first encodes the word vectors in a probabilistic space by a generative TN (word-GTN), and then classifies a text sentence using a discriminative TN (sentence-DTN).  Moreover, in sentence-DTN, its hyper-parameter (i.e., bond-dimension) can be analyzed and selected by the theoretical property of TextTN's expressive power. In experiments, our TextTN also obtains the state-of-the-art result on SST-5 sentiment classification task.