Inducing Target-Specific Latent Structures for Aspect Sentiment Classification (2020)

Abstract: Aspect-level sentiment analysis aims to recognize the sentiment polarity of an aspect or a target in a comment. Recently, graph convolutional networks based on linguistic dependency trees have been studied for this task. However, the dependency parsing accuracy of commercial product comments or tweets might be unsatisfactory. To tackle this problem, we associate linguistic dependency trees with automatically induced aspectspecific graphs. We propose gating mechanisms to dynamically combine information from word dependency graphs and latent graphs which are learned by self-attention networks. Our model can complement supervised syntactic features with latent semantic dependencies. Experimental results on five benchmarks show the effectiveness of our proposed latent models, giving significantly better results than models without using latent graphs.

## introduction aspect-level sentiment analysis aims to classify the sentiment polarities towards specific aspect terms in a given sentence @xcite @xcite . aspects are also called opinion targets, which can typically be product or service features in customer reviews. for example, in the user comment "the environment is romantic, but the food is horrible", the sentiments of the two aspects "environment" and "food" are positive and negative, respectively. the main challenge of aspect-level sentiment analysis is to effectively model the interaction between the aspect and its surrounding contexts. for example, identifying that "romantic" instead of "horrible" as the opinion word is the key to correctly classifying the sentiment of "environment". recently, graph convolutional networks (gcns; @xcite ) over dependency i complained to the manager , but he was not even apologetic (a) an example dependency tree from stanford corenlp parser 2 . the portions are small but being that the food was so good makes up for that . the portions are small but being that the f ood was so good makes up for that . one intuitive solution to the aforementioned problems is to automatically induce semantic structures during the optimization process for sentiment classification. to this end, existing work has investigated latent structures sentence-level sentiment classification @xcite @xcite @xcite , but no existing work has considered aspect-level sentiment classification. for the aspect-level task, a different structure should be ideally learned for each aspect. as shown in figure 1 (b) and 1(c), when given the sentence "the portions are small but being that the food was so good makes up for that.", ideal structures for the aspects "portions" and "food" can consist of links relevant to the terms and their opinion words only, without introducing additional information. we empirically investigate three different methods for inducing semantic dependencies, including attention @xcite , sparse attention @xcite and hard kuma discrete structures @xcite . in particular, attention has been used as a soft alignment structure for tasks such as machine translation @xcite , and sparse attention has been used for text generation @xcite . the hard kumaraswamy distribution has been used to induce discrete structures with full differentiability @xcite . we build a unified self-attentivenetwork (san) framework @xcite for investigating the three structure induction methods, using a graph convolutional network on top of the induced aspect-specific structure for aspect level sentiment classification. in addition, to exploit mutual benefit with dependency syntax, we further consider a novel gate mechanism for merging multiple tree structures during gcn encoding. experiments on five benchmarks including twitter, laptop and restaurant comments show the effectiveness of our proposed latent variable models. our final methods give the state-of-the-art results in the literature, achieving significantly better accuracies than models without using latent graphs. to our knowledge, we are the first to investigate automatically inducing tree structures for targeted sentiment classification. we release our code at https://github.com/ccsoleil/latent graph atsc . ## related work aspect-level sentiment analysis aspect-level sentiment analysis includes three main subtasks, namely aspect term sentiment classification (atsc) @xcite , aspect category sentiment classification (acsc) @xcite @xcite and aspect-term or opinion word extractions @xcite @xcite . in this paper, we focus on atsc. to model relationships between the aspect terms and the context words, @xcite designed target-aware pooling functions to extract discriminative contexts. @xcite modeled the interaction of targets and context words by using target-dependent lstms. @xcite used multi-hop attention and memory networks to correlate an aspect with its opinion words. @xcite design gating mechanisms to select useful contextual information for each target. attention networks are further explored by sequent work @xcite . @xcite al. (2018a) latent graph induction latent graphs can be induced to learn task-specific structures by end-toend models jointly with downstream tasks. @xcite proposed structural attention networks to introduce latent dependency graphs as intermediate layers for neural encoders. @xcite used sparsemap to obtain a sparse distribution over latent dependency trees. @xcite implemented a differentiable proxy to the argmax ## model the overall model structure is shown in figure 1 . the model consists of four main components, including a sequence encoder layer for the input sentence, a structural representation layer that learns a latent induced structure a, a gcn network that represents the latent structure and an aspect oriented classification layer. below we discuss each component in detail in the bottom-up direction. ## experiments we conduct experiments on five benchmark datasets for aspect-level sentiment analysis, including twitter posts (twitter) from stanza @xcite . no dependency labels are used. for the other settings, we follow @xcite . following previous conventions, we repeat each model three times and average the results, reporting accuracy (acc.) and macro-f1 (f1). ## conclusion we considered latent graph structures for aspect sentiment classification by investigating a variety of neural networks for structure induction, and novel gated mechanisms to dynamically combine different structures. compared with dependency tree gcn baselines, the model does not introduce additional model parameters, yet significantly enhances the representation power. experiments on five benchmarks show effectiveness of our model. to our knowledge, we are the first to investigate latent structures for aspect level sentiment classification, achieving the best-reported accuracy on five benchmark datasets.