B i S yn- GAT +: Bi-Syntax Aware Graph Attention Network for Aspect-based Sentiment Analysis (2022)

Abstract: Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task that aims to align aspects and corresponding sentiments for aspect-specific sentiment polarity inference. It is challenging because a sentence may contain multiple aspects or complicated (e.g., conditional, coordinating, or adversative) relations. Recently, exploiting dependency syntax information with graph neural networks has been the most popular trend. Despite its success, methods that heavily rely on the dependency tree pose challenges in accurately modeling the alignment of the aspects and their words indicative of sentiment, since the dependency tree may provide noisy signals of unrelated associations (e.g., the “conj” relation between “great” and “dreadful” in Figure 2). In this paper, to alleviate this problem, we propose a Bi-Syntax aware Graph Attention Network (BiSyn-GAT+). Specifically, BiSyn-GAT+ fully exploits the syntax information (e.g., phrase segmentation and hierarchical structure) of the constituent tree of a sentence to model the sentiment-aware context of every single aspect (called intra-context) and the sentiment relations across aspects (called inter-context) for learning. Experiments on four benchmark datasets demonstrate that BiSyn-GAT+ outperforms the state-of-the-art methods consistently.

## introduction aspect-based sentiment analysis (absa) aims to identify the sentiment polarity towards a given aspect in the sentence. many previous works @xcite mainly focus on extracting sequence features via recurrent neural networks (rnns) or convolution neural networks (cnns) with attention mechanisms, which often assume that words closer to the target aspect are corresponding author nsubj cop cc conj nsubj cop det det cc conj det but the food is great the service and the environment are @xmath0: dependency tree of " the food is great but the service and the environment are dreadful". two separate ellipses encircle its two clauses. the "conj" edge between "great" and "dreadful" is a noise. to mitigate this problem, there already exists several efforts @xcite dedicated to research on how to effectively leverage non-sequential information (e.g., syntactic information like dependency tree) via graph neural networks (gnns). generally, a dependency tree (i.e., dep.tree), linking the aspect terms to the syntactically related words, stays valid in the long-distance dependency problem. however, the inherent nature of dep.tree structure may introduce noise like the unrelated relations across clauses, such as "conj" relation between "great" and "dreadful" in figure 2 , which discourages capturing the sentiment-aware context of each aspect, i.e., intra-context. moreover, the dep.tree structure only reveals relations between words and, thereby, in most cases, is incapable of modeling complicated (e.g., conditional, coordinating, or adversative) relations of sentences, therefore failing to capture sentiment relations between aspects, i.e., inter-context. hence, in this paper, we consider fully exploiting the syntax information of the constituent tree to tackle the problem. typically, a constituent tree (i.e., con.tree) often contains precise and discriminative phrase segmentation and hierarchical composition structure, which are helpful for correctly aligning the aspects and their corresponding words indicative of sentiment. the former can naturally divide a complicated sentence into multiple clauses, and the latter can discriminate different relations among aspects to infer the sentiment relations of different aspects. we illustrate this with an example in figure 3: (1) clause "the food is great" and the clause "the service and environment are dreadful" are segmented by the phrase segmentation term "but"; (2) in layer-1, the term "and" indicates the coordinating relation of "service" and "environment", while the term "but" in layer-3 reflects the adversative relation towards "food" and "service" (or "environment"). thus, to better align aspect terms and corresponding sentiments, we propose a new framework, bi-syntax aware graph attention network (bisyn-gat+), to effectively leverage the syntax information of constituent tree by modeling intracontext and inter-context information. in particular, bisyn-gat+ employs: 1) a syntax graph embedding to encode the intra-context of each aspect based on the fusion syntax information within the same clause in a bottom-up way, which combines the phrase-level syntax information of its constituent tree and the clause-level syntax information of its dependency tree. 2) an aspect-context graph consisting of phrase segmentation terms and all aspects to model the inter-context of each aspect. specifically, it aggregates the sentiment infor-mation of other aspects according to the influence between the current aspect and its neighbor aspects, which is calculated based on aspect representations learned from bi-directional relations over the aspect context graph, respectively. our main contributions are as follows: ## related work sentiment analysis is an important task in the field of natural language processing @xcite and can be applied in downstream tasks, like emotional chatbot @xcite @xcite , recommendation system @xcite , qa system @xcite . here we focus on a fine-grained sentiment analysis task -absa. recently, deep learning methods have been widely adopted for absa task. these works can be divided into two main categories: methods without syntax information (i.e., syntax-free methods) and methods with syntax information (i.e., syntax-based methods). syntax-free methods: neural networks with attention mechanisms @xcite @xcite have been widely used. @xcite adopts a multiple-attention mechanism to capture sentiment features. song et al. (2019) uses an attentional encoder network (aen) to excavate rich semantic information from word embeddings. syntax-based methods: recently, utilizing dependency information with gnns has become an effective way for absa. zhang et al. (2019) uses graph convolutional networks (gcn) to learn node representations from dep.tree. tang et al. (2020) proposes a dependency graph enhanced dual-transformer network (dgedt) by jointly considering representations from transformers and corresponding dependency graph. wang et al. (2020a) constructs aspect-oriented dependency trees and proposes r-gat, extending the graph attention network to encode graphs with labeled edges. li et al. (2021) proposes a dual graph convolutional networks (dualgcn) model, simultaneously considering syntax structures and semantic correlations. all above works use syntax information of dep.tree, which may introduce noise, as we said before. thus, we exploit syntax information of con.tree with gnns. precisely, we follow the con.tree to aggregate information from words within the same phrases in a bottom-up way and capture intra-context information. moreover, some works resort to modeling aspectaspect relations. some @xcite adopt aspect representations to model relations by rnns or memory networks, without utilizing context information. and some @xcite propose alignment loss or orthogonal attention regulation to constrain aspect-level interactions, which fail when aspects have no explicit opinion expressions or multiple aspects share same opinion words. recently, there are some works utilizing gnns to model aspect relations. @xcite constructs an inter-aspect graph based on relative dependencies between aspects. @xcite constructs a sentiment graph, where each node represents an aspect, and each edge represents the sentiment dependency relation. however, these works fail to explicitly use phrase segmentation information, such as conjunction words. thus, we propose an aspect-context graph consisting of all aspects and phrase segmentation terms to model inter-context information. gnns with constituent tree: to our knowledge, we are the first work to utilize the constituent tree for absa task. but in aspect-category sentiment analysis task, which predicts sentiment polarity towards a given predefined category in the text, @xcite proposes a sentence constituent-aware network (scan) that generates representations of the nodes in con.tree. unlike scan, we view parsed phrases as different spans of the input text instead of individual nodes. so we don't introduce any inner nodes of con.tree (e.g., "np","vp" of figure 3 ) into the representation space, decreas-ing the computational overhead. architecture. as shown in figure 4 , our proposed architecture takes the sentence and all aspects that appear in the text as the input, and outputs the sentiment predictions of the aspects. it contains three components: 1) the intra-context module encodes the input {w i } to obtain aspect-specific representations of the target aspects, which contains two encoders: a context encoder that outputs contextual word representations and a syntax encoder that utilizes syntax information of the parsed constituent tree (or, and dependency tree). 2) the inter-context module includes a relation encoder applied to the constructed aspect-context graph to output relation-enhanced representations. the aspect-context graph composes all aspects of the given sentence and phrase segmentation terms obtained from a designed rule-based map function applied to the constituent tree. 3) the sentiment classifier takes output representations of the above two modules to make predictions. ## effects of aspect-context graph we also investigate the effects of our bi-relational modeling of the proposed aspect-context graph. one possible reason is that the number of samples that contain at least three aspects is very limited, as shown in ## conclusion in this paper, we propose the bisyn-gat+ framework to model the sentiment-aware context of each aspect and sentiment relations across aspects for learning by fully exploiting the syntax information of the constituent tree. it includes two welldesigned modules: 1) intra-context module that fuses related semantic and syntax information hierarchically; 2) inter-context module that models relations across aspects with the constructed aspectcontext graph. to the best of our knowledge, it is the first work to exploit the constituent tree with gnns for the absa task. moreover, our proposed model achieves state-of-the-art performances on four benchmark datasets.