AMR -based Network for Aspect-based Sentiment Analysis (2023)

Abstract: Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment classification task. Many recent works have used dependency trees to extract the relation between aspects and contexts and have achieved significant improvements. However, further improvement is limited due to the potential mismatch between the dependency tree as a syntactic structure and the sentiment classification as a semantic task. To alleviate this gap, we replace the syntactic dependency tree with the semantic structure named Abstract Meaning Representation (AMR) and propose a model called AMR-based Path Aggregation Relational Network (APARN) to take full advantage of semantic structures. In particular, we design the path aggregator and the relation-enhanced self-attention mechanism that complement each other. The path aggregator extracts semantic features from AMRs under the guidance of sentence information, while the relation-enhanced self-attention mechanism in turn improves sentence features with refined semantic information. Experimental results on four public datasets demonstrate 1.13% average F1 improvement of APARN in ABSA when compared with state-of-the-art baselines.

## introduction recent years have witnessed growing popularity of the sentiment analysis tasks in natural language processing @xcite . aspect-based sentiment analysis (absa) is a finegrained sentiment analysis task to recognize the sentiment polarities of specific aspect terms in a given sentence @xcite @xcite . for example, here is a restaurant review "all the money went into the interior decoration, none of it went to the chefs" and the sentiment polarity of two aspects we were amazed at how small the dish was. "interior decoration" and "chefs" are positive and negative, respectively. thus, absa can precisely recognize the corresponding sentiment polarity for any aspect, different from allocating a general sentiment polarity to a sentence in sentence-level sentiment analysis. the key challenge for absa is to capture the relation between an aspect and its context, especially opinion terms. in addition, sentences with multiple aspects and several opinion terms make the problem more complex. to this end, some previous studies @xcite @xcite @xcite have devoted the main efforts to attention mechanisms. despite their achievements in aspect-targeted representations and appealing results, these methods always suffers noise from the mismatching opinion terms or irrelevant contexts. on the other hand, more recent studies @xcite @xcite propose models explicitly exploit dependency trees, the syntactic structure of a sentence, to help attention mechanisms more accurately identify the interaction between the aspect and the opinion expressions. these models usually employ graph neural networks over the syntactic dependencies and display significant effectiveness. however, existing absa models still indicate two potential limitations. first, there appears to be a gap between the syntactic dependency structure and the semantic sentiment analysis task. considering the sentence in figure 1 , "small" semantically modifies "dish" and expresses negative sentiment, but both "small" and "dish" are syntactically dependent on "was". the determinant of sentiment should be the meaning of the sentence rather than the way it is expressed. second, the output of natural language parsers including dependency parsers always contains inaccuracies @xcite . without further adjustment, raw results of parsers can cause errors and be unsuitable for absa task. to solve aforementioned challenges, we propose a novel architecture called amr-based path aggregation relational network (aparn). for the first challenge, we introduce abstract meaning representations (amrs), a powerful semantic structure. for the amr example in figure 1 , "small" and "dish" are directly connected, while function words such as "were" and "at" disappear, which makes it easier to establish the aspect-opinion connection and shows the advantage of amrs in absa. for the second challenge, we construct the path aggregator and the relation-enhanced self-attention mechanism. the path aggregator integrates the information from amrs and sentences to obtain optimized relational features. this procedure not only encourages consistency between semantic structures and basic sentences, but also achieves the global feature by broadcasting local information along the path in the graph. relation-enhanced self-attention mechanism then adds these relational feature back into attention weights of word features. thanks to these modules, aparn acquires to utilize sentences and amrs jointly and achieves higher accuracy on sentiment classification. to summarize, our main contributions are highlighted as follows: analytical experiments further verify the significance of our model and the amr. ## parsed structures we perform some experiments and discussions for the characteristics of amr compared to parsing structures already used for the absa task and how these characteristics affect our aparn. human-defined structures dependency trees and amrs are parsed based on human-defined syntactic and semantic rules, respectively. each word in a sentence becomes a node of the dependency tree, but in the amr, relational words like function words and auxiliary words are represented as edges, while concept words like nouns and verbs are refined into nodes in the graph. with amr aligning, we can map concept words in sentences to nodes in the graph and establish relations between them, while relation words are isolated. to estimate the impact of dependency trees and amrs in the absa task, we calculate the average distance between aspect words and opinion words in different parsed structures on the restaurant dataset, called aspect-opinion distance (aod). we also calculate the average distance between aspect words and all context words called aspectcontext distance (acd), and divide aod by acd as relative aspect-opinion distance (raod). the distance between aspect words and isolated words is treated as sentence length. according to the result shown in the above analysis is for graph skeletons, and we also explore the impact of edge labels of two structures in the absa task. figure 2 compares the distribution of edge labels in aspect-opinion paths with the distribution of all edge labels. these distributions are clearly different, both in dependency trees and amrs, which implies that edge labels can also help the absa task, especially in amrs. based on these characteristics, we design the outer product sum module for aparn to mix sentence information into the graph, and design the path aggregator to collect graph skeleton and edge label information in amrs. data-driven structures some existing studies use structures produced by data-driven models in the absa task @xcite @xcite and exhibit different effects from human-defined structures. therefore, we design a relation-enhanced self-attention mechanism for aparn to integrate the graph information obtained by the path aggregator with the information from the pre-trained model. ## proposed model the overall architecture of our proposed model aparn is illustrated in figure 3 . it consists of 3 parts: amr preprocessing, path aggregator and relation-enhanced self-attention mechanism. in the absa task, a sentence @xmath0, w 2 , ..., w n } and a specific aspect term @xmath1, a 2 , ..., a m } are given to determine the corresponding sentiment polarity class c a , where a is a sub-sequence of s and c @xmath2, n eutral, n egative}. many existing works use syntactic dependency trees to establish explicit or implicit connections between aspects and contexts. however, we believe that the sentiment analysis task is essentially about the meanings of sentences, so semantic structures like amrs are more favorable for this task. in addition, amrs are more concise than dependency trees, making it easier to extract valuable information in training but more difficult to preprocess before training. we have to conduct a series of steps including: amr parsing, amr aligning and amr embedding. preprocessed amrs still have errors and unsuitable parts for the task, so we design the path aggregator and the relation-enhanced self-attention mechanism to perform joint representation refinement and flexible feature fusion on the amr graph and the original sentence. next, we elaborate on the details of our proposed aparn, including amr preprocessing and embedding, the path aggregator and the relationenhanced self-attention mechanism. ## experiments in this section, we first introduce the relevant settings of the experiments, including the datasets used, implementation details and baseline methods for comparison. then, we report the experimental results under basic and advanced settings. finally, we select several representative examples for model analysis and discussion. ## related work aspect-based sentiment analysis traditional sentiment analysis tasks are usually sentence-level or document-level, while the absa task is an entity-level and fine-grained sentiment analysis task. early methods @xcite are mostly based on artificially constructed features, which are difficult to effectively model the relations between aspect terms and its context. with the development of deep neural networks, many recent works @xcite @xcite @xcite @xcite have explored applying attention mechanisms to implicitly model the semantic relations of aspect terms and identify the key opinion terms in the context. another trend in absa studies is the explicit use of dependency trees. some works @xcite @xcite @xcite @xcite @xcite @xcite extend gcn, gat, and transformer backbones to process syntactic dependency trees and develop several outstanding models. these models shorten the distance between aspect terms and opinion terms by dependency trees and alleviate the long-term dependency problem. recent studies have also noticed the limitations of dependency trees in the absa task. @xcite proposes the reshaped dependency tree for the absa task. @xcite propose to combine dependency trees with induced aspect-specific latent maps. @xcite further proposed an aspect-specific and language-independent discrete latent tree model as an alternative structure for dependency trees. our work is similar in that we also aim at the mismatch between dependency trees and the absa task, but different in that we introduce a semantic structure amr instead of induced trees. abstract meaning representation amr is a structured semantic representation that represents the semantics of sentences as a rooted, directed, acyclic graph with labels on nodes and edges. amr is proposed by @xcite to provide a specification for sentence-level comprehensive semantic annotation and analysis tasks. research on amr can be divided into two categories, amr parsing @xcite @xcite and amr-to-text @xcite @xcite . amr has also been applied in many nlp tasks. @xcite use amr in question answering system. @xcite employ amr to improve common sense reasoning. @xcite utilize amr to add pseudo labels to unlabeled data in low-resource event extraction task. our model also improves the performance of the absa task with amr. moreover, amr also has the potential to be applied to a broader range of nlp tasks, including relation extraction @xcite , named entity recognition @xcite , natural language inference @xcite ), text-to-sql @xcite , and more. ## conclusion in this paper, we propose aparn, amr-based path aggregation relational network for absa. different from the traditional absa model utilizing the syntactic structure like dependency tree, our model employs the semantic structure called abstract meaning representation which is more harmony with the sentiment analysis task. we propose the path aggregator and the relation-enhanced selfattention mechanism to efficiently exploit amrs and integrate information from amrs and input sentences. these designs enable our model to achieve better results than existing models. experiments on four public datasets show that aparn outperforms competing baselines.