Quantum-inspired Language Model with Lindblad Master Equation and Interference Measurement for Sentiment Analysis (2024)

Abstract: Quantum-inspired models have demonstrated superior performance in many downstream language tasks, such as question answering and sentiment analysis. However, recent models primarily focus on embedding and measurement operations, overlooking the significance of the quantum evolution process. In this work, we present a novel quantum-inspired neural network, LI-QiLM, which integrates the Lindblad Master Equation (LME) to model the evolution process and the interferometry to the measurement process, providing more physical meaning to strengthen the interpretability. We conduct comprehensive experiments on six sentiment analysis datasets. Compared to the traditional neural networks, transformer-based pre-trained models and quantum-inspired models, such as CICWE-QNN and ComplexQNN, the proposed method demonstrates superior performance in accuracy and F1-score on six commonly used datasets for sentiment analysis. Additional ablation tests verify the effectiveness of LME and interferometry.

## introduction the rapid advancement of deep learning has significantly boosted the performance of natural language processing (nlp) tasks. however, a key limitation of deep learning models is their lack of interpretability @xcite . recently, there has been a promising trend toward harnessing quantum theory to augment expressive capabilities and enhance interpretability @xcite . the similarities between quantum systems and natural languages have been widely explored @xcite . in linguistics, a word is given different meanings. such polysemy phenomenon is analogously compared to the superposition state in quantum mechanics. in this sense, sentences are analogous to a multiparticle system. besides, as * corresponding author particles in quantum mechanics collapse to defined states upon observation, language users assign specific meanings to words or sentences based on contexts. meanwhile, studies @xcite conducted within the realm of cognitive science have demonstrated that human cognition, encompassing sentiment analysis tasks in nlp, often exhibit quantum-like behaviors, which can be more precisely elucidated through quantum theory. building upon these foundational ideas, quantum-inspired language models (qilms) are actively explored and designed to augment the performance of traditional language models @xcite @xcite . a quantum-inspired multimodal sentiment analysis framework named qmsa @xcite applied interference experiments to sentiment analysis by comparing emotion labels to photons and data types to interference slits. motivated by the complex word embedding, an embedding method called icwe @xcite is proposed to use the amplitude and phase of quantum as the real and imaginary part of the word embedding respectively, which improves the feature extraction ability of the model. a quantum-inspired multimodal fusion framework @xcite converts real-valued inputs of different modalities into complex-valued pure quantum states and induces the different modalities to interact in a non-separable way for encoding cross-modal information. their study showcased the superior performance of this novel approach compared to conventional transformer-based models @xcite . this groundbreaking fusion of domains is expected to make significant contributions to the evolution of quantum machine learning and play a pivotal role in practical use. the lindblad master equation (lme) (manzano, 2020), a crucial instrument in the exploration of open quantum systems, remains neglected by contemporary models. it stands as an indispensable complement to the schrödinger equation, providing a highly versatile framework for depicting markovian dynamics in quantum systems. specifically, this equation provides a comprehensive portrayal of the temporal evolution of the density matrix, incorporating the interactions with the surrounding environment. taking advantage of lme, a more appropriate density matrix representation can be learned for language sentences in an end-to-end manner. simultaneously, current models usually use projection measurement, which demands an extensive amount of operations. this is because the probability distribution on each basis is derived through statistical averaging, leading to an augmented time expenditure and uncertainty. in contrast, interferometry solely necessitates the utilization of an interferometric operator to imitate the collapse procedure of the superposition state. consequently, it swiftly attains dependable outcomes with enhanced precision. in this work, we propose a quantum-inspired language model with lindblad master equation and interferometry (li-qilm) to introduce a robust quantum evolution process, a more accurate measurement process, and an improved expressive capacity through the inclusion of additional learnable parameters. our contributions are: we organize the paper as follows. section 2 presents the fundamentals of quantum theory. section 3 provides an overview of related works. in section 4, we introduce the proposed model. section 5 details the experiments carried out. in section 6, we draw our conclusions. additionally, section 7 discusses the limitations of our work. ## preliminary quantum computing represents a revolutionary computing paradigm that follows the laws of quantum mechanics to regulate quantum information units to calculate. in the open quantum system, it is imperative to employ the density matrix as a means to accurately portray the system's state. delving into the system's evolutionary process necessitates the utilization of the quantum master equation. ultimately, precise quantum measurement techniques allow us to obtain unequivocal results regarding the system's evolution. the specific process is shown in fig. 1 . ## related works the development of quantum computing has been ongoing since 1982 @xcite . with the surge in machine learning, the integration of machine learning and quantum theory has drawn closer attention. a pivotal landmark in the field of nlp was the introduction of the quantum language model by @xcite . @xcite , which applied the probabilistic framework of quantum theory to information retrieval tasks. this model utilized the density matrix to create a nuanced text representation, showing substantial gains in performance compared to the traditional bag-of-words models @xcite . @xcite . @xcite introduced the neural network based quantum-like language model (nnqlm), marking a notable advancement in the field. nnqlm pioneers a unique strategy with the use of a tailored density matrix that ingeniously captures the relationship between questions and answers, significantly improving the representation of sentence similarity. it is seamlessly integrated into a two-dimensional neural network architecture designed for matching. @xcite . @xcite noticed a striking parallel between the way a word's meaning is influenced by its companions and the behavior patterns of microscopic particles. drawing from this observation, they proposed a complex embedding mixture network (ce-mix) that employs word embedding in the complex domain. ce-mix deduces the significance of phrases using the principle of quantum interference. in evaluations, it surpassed the leading non-quantum models in sentence classification tasks. @xcite . @xcite introduced the convolutional complex-valued neural network based on interpretable complex-valued word embedding (cicwe-qnn). this model utilizes the amplitude and phase of quantum to respectively represent the real and imaginary parts of word embeddings. this approach improves feature extraction capabilities while also guaranteeing the model's interpretability. @xcite . @xcite proposed a complex-valued neural network-based qlm (c-nnqlm), which employs an end-to-end methodology for constructing and refining density matrices in a differentiable way. this approach facilitates the incorporation of well-trained word vectors and supervised labels. furthermore, c-nnqlm incorporates complex value word embedding, with its phase vector adeptly encoding the sequential or positional information of words. @xcite . @xcite presented the quantum-inspired fully complex-valued neutral network, complexqnn. this model employs complex-value cnn throughout the processing pipeline, aligning more closely with quantum computing theory and offering the potential for future acceleration on quantum computers. in sharp contrast to conventional neural networks, quantum-inspired models present the distinct advantage of offering explanations grounded in physical significance. furthermore, they exhibit the unique capability to incorporate microscopic particle phenomena into the model, which serves to augment their learning capacity. however, the quantum evolution process is rarely considered in recent quantum-inspired models and projection measurements require more actions to ascertain the probability distribution. therefore, in this work, we focus on that and propose to use lme in the learning process and interferometry in the measurement process. ## method fig. 2 shows the proposed framework, which consists of the alterations and operations performed on the data at every stage of the process. ## conclusions in this study, we propose li-qilm, a quantuminspired neural network, built on quantum evolution and interferometry. we have provided detailed principles, highlighting key implementation modules, including the complex-valued inception module, lme module and interferometry module. furthe best score of each dataset is in bold. thermore, we conducted evaluations of li-qilm on six varied datasets and the proposed model consistently outperformed the existing popular methods. finally, ablation tests were conducted to appraise the effectiveness of the key modules. in the future, we will aim to augment the capacity of the model to extract profound linguistic insights by integrating dynamic word embedding. furthermore, few of the current qilms have quantitatively evaluated the effectiveness of the quantum concepts employed within them. therefore, greater efforts must be directed towards assessing the efficacy of various components within qilms.