Aspect Sentiment Classification with Document-level Sentiment Preference Modeling (2020)

Abstract: In the literature, existing studies always consider Aspect Sentiment Classification (ASC) as an independent sentence-level classification problem aspect by aspect, which largely ignore the document-level sentiment preference information, though obviously such information is crucial for alleviating the information deficiency problem in ASC. In this paper, we explore two kinds of sentiment preference information inside a document, i.e., contextual sentiment consistency w.r.t. the same aspect (namely intra-aspect sentiment consistency) and contextual sentiment tendency w.r.t. all the related aspects (namely inter-aspect sentiment tendency). On the basis, we propose a Cooperative Graph Attention Networks (CoGAN) approach for cooperatively learning the aspect-related sentence representation. Specifically, two graph attention networks are leveraged to model above two kinds of document-level sentiment preference information respectively, followed by an interactive mechanism to integrate the two-fold preference. Detailed evaluation demonstrates the great advantage of the proposed approach to ASC over the state-of-the-art baselines. This justifies the importance of the document-level sentiment preference information to ASC and the effectiveness of our approach capturing such information.

## introduction aspect sentiment classification (asc), a finegrained sentiment classification task in the field of sentiment analysis @xcite , aims to identify the sentiment polarity (e.g., positive, negative or neutral) for each aspect discussed inside a sentence. for example, the sentence "the restaurant has quite low price but the food tastes not good" would be assigned with a positive polarity for the aspect price and with a negative * corresponding author: jingjing wang. @xcite ) datasets, where aspect category is defined as the entity e and attribute a pair (i.e., e#a). red lines denote the intra-aspect sentiment consistency and blue lines denote the inter-aspect sentiment tendency. polarity for the aspect food. over the past decade, the asc task has been drawing more and more interests @xcite due to its wide applications, such as e-commerce customer service @xcite , public opinion mining @xcite and question answering @xcite . ## related work in this section, we first review the aspect sentiment classification (asc) task, and then introduce the related studies on graph-based neural networks. aspect sentiment classification. the asc task aims to predict the sentiment polarity for each aspect discussed inside a sentence. existing studies mainly focus on utilizing various approaches (e.g., attention mechanism and memory network) to align each aspect and the sentence for learning aspect-related sentence representation. @xcite propose an attention-based lstm in order to explore the potential correlation of aspects and sentiment polarities in asc. @xcite propose a hierarchical attention network to incorporate both words and clauses information for asc. @xcite propose an attention-based approach to incorporate the aspect-related syntactic information for asc. @xcite and @xcite design deep memory networks to align the aspect and sentence for asc. @xcite propose a semantic and context-aware memory network to integrate aspect-related semantic parsing information for performing asc. @xcite and @xcite leverage reinforcement learning grounded approaches to select aspect-relevant words for asc. recently, a few studies have recognized the information deficiency problem in asc and attempted to using external information to improve the performance of asc. @xcite and @xcite incorporate the knowledge from document-level sentiment classification to improve the performance of asc. @xcite propose an extension of lstm to integrate the commonsense knowledge into the recurrent encoder for improving the performance of asc. in addition, it is worthwhile to note that hazarika et al. (2018) also investigate the inter-aspect sentiment dependency for asc, but is limited to capture this information inside a single sentence. in summary, all the above studies ignore the document-level sentiment preference information, which can be leveraged to effectively mitigate the information deficiency problem in asc. graph-based neural networks. in recent years, graph-based neural networks have received more and more attentions. as a pioneer, @xcite present a simplified graph neural network model, called graph convolutional networks (gcn), which has been exported to several tasks such as scene recognition @xcite , semi-supervised node classification @xcite ), text-to-sql parsing @xcite and relation extraction @xcite . on this basis, some other improved graph-based neural networks are proposed. @xcite propose a generalization of graph-based neural networks, so-called k-dimensional gnns (k-gnns), which can take higher-order graph structures at multiple scales into account. @xcite propose a novel multi-channel graph neural network model to learn alignment-oriented knowledge graph embeddings by robustly encoding two knowledge graphs via multiple channels. more recently, there exist several studies also adopting graph-based neural networks to asc. for instance, hou et al. (2019) and zhang et al. (2019a) build gcn over the dependency tree of a sentence to exploit syntactical information and word dependencies for learning better aspect-related sentence representation for asc. different from all the above studies, this paper proposes a novel cooperative graph attention networks approach to capture the document-level sentiment preference information in asc. to our best knowledge, this is the first attempt to incorporate this information for the asc task. ## cooperative graph attention networks (cogan) in this section, we formulate the aspect sentiment classification (asc) task as follows. in each document d with sentences foot_0 {s 1 , s 2 , ..., s i }, given a sentence s i , @xmath0, 2, ..., i} and its aspect a k , @xmath1, 2, ..., k}, the asc task aims to predict the sentiment polarity ℓ for aspect a k by automatically learning the aspect-related sentence representation r i of sentence s i . here, i is the number of sentence s i , and k is the number of aspect a k inside the document. in this paper, we propose a cooperative graph attention networks (cogan) approach with two types of graph attention networks (gan) to incorporate the two-fold preference information respectively. figure 2 shows the overall architecture of the cogan approach which consists of five major blocks: 1) encoding block; 2) intra-aspect consistency modeling block; 3) inter-aspect tendency modeling block; 4) interaction block. 5) softmax decoding block. before introducing our cogan approach, we first give an overview of the basic graph attention network (gan). ## conclusion in this paper, we propose a novel cooperative graph attention networks (cogan) approach to aspect sentiment classification (asc). the main idea of the proposed approach is to incorporate two kinds of sentiment preference information (i.e., the intra-aspect consistency and inter-aspect tendency) in a document for remedying the information deficiency problem in asc. experimental results on four datasets from semeval-2015 and 2016 demonstrate that our approach significantly outperforms a number of competitive baselines, including all the three best-performed systems in the shared tasks of both semeval-2015 and 2016. in our future work, we would like to improve the performance of the asc task by using unlabeled data since our graph-based neural network approach is easy to add unlabeled data. moreover, we would like to apply our approach to other sentiment analysis tasks, e.g., aspect-oriented opinion summarization and multi-label emotion detection.