Discrete Opinion Tree Induction for Aspect-based Sentiment Analysis (2022)

Abstract: Dependency trees have been intensively used with graph neural networks for aspect-based sentiment classification. Though being effective, such methods rely on external dependency parsers, which can be unavailable for low-resource languages or perform worse in low-resource domains. In addition, dependency trees are also not optimized for aspect-based sentiment classification. In this paper, we propose an aspect-specific and language-agnostic discrete latent opinion tree model as an alternative structure to explicit dependency trees. To ease the learning of complicated structured latent variables, we build a connection between aspect-to-context attention scores and syntactic distances, inducing trees from the attention scores. Results on six English benchmarks and one Chinese dataset show that our model can achieve competitive performance and interpretability.

## introduction aspect-based sentiment classification (absa) is the task of recognizing the sentiment polarities of specific aspect categories or aspect terms in a given sentence @xcite @xcite @xcite @xcite @xcite . different from document-level sentiment analysis, different aspect terms in the same document can bear different sentiment polarities. for example, given a restaurant review "decor is nice though service can be spotty", the corresponding sentiment labels of "decor" and "service" are positive and negative, respectively. how to locate the corresponding opinion contexts for each aspect term is a key challenge for absa. to this end, recent efforts leverage dependency trees @xcite @xcite . syntactic dependencies have been shown to better capture the interaction between the aspect and the opinion contexts (huang decor is be nice though service can spotty (a) dependency tree. figure 1 : a dependency tree of the input sentence "decor is nice though service can be spotty" and two induced trees of two aspects in this sentence. in this paper, we explore a simple method to induce a discrete opinion tree structure automatically for each aspect. two examples are shown in figure 1 . in particular, given a target and a sentence, our algorithm induces a tree structure recursively according to a set of attention scores, calculated using a neural layer on top of bert representation of the sentence @xcite . starting with the root node, the algorithm builds a tree by selecting one child node on each side of a current node and recursively continue the partition process to obtain a binarized and lexicalized tree structure. the resulting tree serves as the input structure and is fed into graph convolutional networks @xcite for learning the sentiment classifier. we study policy-based reinforcement learning @xcite to train the tree inducer. one challenge is that the generated policy can be easily remembered by the bert encoder, which leads to insufficient explorations @xcite . to alleviate this issue, we propose a set of regularizers to help bert-based policy generations. although our method is conceptually simple and straightforward for the inference stage, we show that it has a deep theoretic grounding. in particular, the attention based tree induction parsers trained using the policy network can be viewed as a simplified version to a standard latent tree structured vae model @xcite , where the kl divergence between the prior and the posterior tree probabilities is approximated by attention-based syntactic distance measures @xcite . experiments on six english benchmarks, a chinese hotel review dataset and a korean automotive review dataset show the effectiveness of our proposed models. the discrete structure also makes it easy to interpret the classification results. in addition, our algorithm is faster, smaller and more accurate than a full variational latent tree variable model. to our knowledge, we are the first to learn aspect-specific discrete opinion tree structures with bert. we make our code publicly available at https://github.com/ccsoleil/dotgcn . ## model figure 2 shows the architecture of our proposed model. given an input sentence x and a specific aspect term a, we induce an opinion tree t according to a recognition network q ϕ (t|x, a), where ϕ is the set of network parameters. we apply multi-layered graph convolutional networks (gcns) over the bert output vectors to model the structural relations in the opinion tree and extract aspect-specific features. finally, we use an attention-based classifier to learn the sentiment classifier p θ (y|x, a, t), where θ is the set of parameters. to train the model, rl is used for q ϕ (t|x, a) (section 2.3) and standard backpropagation is used for training p θ (y|x, a, t) (section 2.2). ## a variational inference perspective interestingly, l sup , l rl and l att can be unified in a theoretic framework using variational inference (kingma and welling, 2014). we show in this section, that our method can be viewed as a stronger extension to a latent tree vae model. ## experiments we perform experiments on eight aspect-based sentiment analysis benchmarks, including six english datasets, one chinese dataset, and one korean datase. the data statistics is shown in appendix a.3. we use stanza @xcite as the external parser to produce dependency parses for comparing with dependency tree based models, reporting accuracy (acc.) and macro-f1 (f1) scores for each model. more details are presented in appendix a.1. @xcite provide a recent challenge dataset with 4,297 sentences and 11,186 aspects. we take it as the main dataset because it is a large-scale multi-aspect dataset with more aspects in each sentence compared to the other datasets. ## related work tree induction for absa there has been much work on unsupervised discrete induction @xcite @xcite @xcite @xcite , which aims to obtain general constituent trees without explicit syntax annotations and task-dependent supervised signals. we focus on learning task-specific tree-structures for absa, where the tree is fully binarized and lexicalized. @xcite propose gumbel tree-lstm for learning task-specific tree for semantic compositions. similarly, @xcite propose an unsupervised chart parser for jointly learning sentence embeddings and syntax. however, they focus on sentence-level tasks and do not consider aspect information. aspect-level sentiment classification much recent work has explored neural attention mechanism to this task @xcite @xcite 2020 ) use duel-transformer structure to enhance the depen-dency graph for this task. our work is similar in that we also consider the structure dependencies, but different in that we rely on automatically induced tree structures instead of external parses. @xcite propose to induce aspect-specific latent graph by sampling from self-attention-based hard kumaraswamy distributions @xcite . however, to achieve competitive performance, their method still requires a combination of external dependency parse trees and the induced latent graphs. sun et al. (2019a) and xu et al. (2019) constructed aspect related auxiliary sentences as inputs to bert @xcite for strong contextual encoders. @xcite proposed bert-based post training for enhancing domain-specific contextual representations for aspect sentiment analysis. our work shares a similar feature extraction approach, but differently we focus on inducing latent trees for absa. ## conclusion we proposed a method to induce aspect-specific discrete opinion trees for aspect-based sentiment analysis, obtaining trees by viewing aspect-to-context attention scores as syntactic distances. the attention scores are trained using both rl and a novel attention-based regularization. our model empirically achieves competitive performance compared with dependency tree based models, while being independent of parsers. we also provide a theoretic view of our method using variational inference.