AMR-based Network for Aspect-based Sentiment Analysis
Year: 2023

Abstract:
Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment classification task. Many recent works have used dependency trees to extract the relation between aspects and contexts and have achieved significant improvements. However, further improvement is limited due to the potential mismatch between the dependency tree as a syntactic structure and the sentiment classification as a semantic task. To alleviate this gap, we replace the syntactic dependency tree with the semantic structure named Abstract Meaning Representation (AMR) and propose a model called AMR-based Path Aggregation Relational Network (APARN) to take full advantage of semantic structures. In particular, we design the path aggregator and the relation-enhanced selfattention mechanism that complement each other. The path aggregator extracts semantic features from AMRs under the guidance of sentence information, while the relationenhanced self-attention mechanism in turn improves sentence features with refined semantic information. Experimental results on four public datasets demonstrate 1.13% average F1 improvement of APARN in ABSA when compared with state-of-the-art baselines.1

heading: 1 Introduction
text: Recent years have witnessed growing popularity of the sentiment analysis tasks in natural language processing (Li and Hovy, 2017; Birjali et al., 2021). Aspect-based sentiment analysis (ABSA) is a finegrained sentiment analysis task to recognize the sentiment polarities of specific aspect terms in a given sentence (Jiang et al., 2011; Li et al., 2018; Seoh et al., 2021; Zhang et al., 2022a). For example, here is a restaurant review “All the money went into the interior decoration, none of it went to the chefs” and the sentiment polarity of two aspects
1The code will be available at https://github.com/THUBPM/APARN.
*Corresponding Author.
amaze -01
we dish
so
ARG1
ARG0
howatamazed thewereWe dish was cop
nsubj mark
advcl
small
small
dep advmod det nsubj
domain
degree
Dependency Tree:
AMR:
Sentence: We were amazed at how small the dish was.
Figure 1: Comparison of the dependency tree and the AMR. The aspect is red and the opinion term is blue.
“interior decoration” and “chefs” are positive and negative, respectively. Thus, ABSA can precisely recognize the corresponding sentiment polarity for any aspect, different from allocating a general sentiment polarity to a sentence in sentence-level sentiment analysis.
The key challenge for ABSA is to capture the relation between an aspect and its context, especially opinion terms. In addition, sentences with multiple aspects and several opinion terms make the problem more complex. To this end, some previous studies (Wang et al., 2016; Chen et al., 2017; Gu et al., 2018; Du et al., 2019; Liang et al., 2019; Xing et al., 2019) have devoted the main efforts to attention mechanisms. Despite their achievements in aspect-targeted representations and appealing results, these methods always suffers noise from the mismatching opinion terms or irrelevant contexts.
On the other hand, more recent studies (Zhang et al., 2019a; Tang et al., 2020; Li et al., 2021; Xiao et al., 2021) propose models explicitly exploit dependency trees, the syntactic structure of a sentence, to help attention mechanisms more accurately identify the interaction between the aspect and the opinion expressions. These models usually employ graph neural networks over the syntactic dependencies and display significant effectiveness. However, existing ABSA models still indicate two potential limitations. First, there appears to be a
322
gap between the syntactic dependency structure and the semantic sentiment analysis task. Considering the sentence in Figure 1, “small” semantically modifies “dish” and expresses negative sentiment, but both “small” and “dish” are syntactically dependent on “was”. The determinant of sentiment should be the meaning of the sentence rather than the way it is expressed. Second, the output of natural language parsers including dependency parsers always contains inaccuracies (Wang et al., 2020). Without further adjustment, raw results of parsers can cause errors and be unsuitable for ABSA task.
To solve aforementioned challenges, we propose a novel architecture called AMR-based Path Aggregation Relational Network (APARN). For the first challenge, we introduce Abstract Meaning Representations (AMRs), a powerful semantic structure. For the AMR example in Figure 1, "small" and "dish" are directly connected, while function words such as "were" and "at" disappear, which makes it easier to establish the aspect-opinion connection and shows the advantage of AMRs in ABSA. For the second challenge, we construct the path aggregator and the relation-enhanced self-attention mechanism. The path aggregator integrates the information from AMRs and sentences to obtain optimized relational features. This procedure not only encourages consistency between semantic structures and basic sentences, but also achieves the global feature by broadcasting local information along the path in the graph. Relation-enhanced self-attention mechanism then adds these relational feature back into attention weights of word features. Thanks to these modules, APARN acquires to utilize sentences and AMRs jointly and achieves higher accuracy on sentiment classification.
To summarize, our main contributions are highlighted as follows:
• We introduce Abstract Meaning Representations into the ABSA task. As a semantic structure, the AMR is more suitable for sentiment analysis task. • We propose a new model APARN that integrates information from original sentences and AMRs via the path aggregator and the relation-enhanced self-attention mechanism to fully exploit semantic structure information and relieve parser unreliability. • We experiment on four public datasets and our APARN outperforms state-of-the-art baselines, demonstrating its effectiveness. More
analytical experiments further verify the significance of our model and the AMR.

heading: 2 Parsed Structures
text: We perform some experiments and discussions for the characteristics of AMR compared to parsing structures already used for the ABSA task and how these characteristics affect our APARN.
Human-defined Structures Dependency trees and AMRs are parsed based on human-defined syntactic and semantic rules, respectively. Each word in a sentence becomes a node of the dependency tree, but in the AMR, relational words like function words and auxiliary words are represented as edges, while concept words like nouns and verbs are refined into nodes in the graph. With AMR aligning, we can map concept words in sentences to nodes in the graph and establish relations between them, while relation words are isolated.
To estimate the impact of dependency trees and AMRs in the ABSA task, we calculate the average distance between aspect words and opinion words in different parsed structures on the Restaurant dataset, called aspect-opinion distance (AOD). We also calculate the average distance between aspect words and all context words called aspectcontext distance (ACD), and divide AOD by ACD as relative aspect-opinion distance (rAOD). The distance between aspect words and isolated words
is treated as sentence length. According to the result shown in Table 1, both dependency trees and AMRs have similar AOD smaller than original sentences, which indicates their benefits to capture relations about aspects. Due to the elimination of isolated words, the rAOD of AMRs is much less than dependency trees, which means smaller scope and easier focus. About 2.13% of opinion words are wrongly isolated, making the AOD of AMR (all words) a little bigger. But this is acceptable considering the improvement of rAOD and partially repairable by information from original sentences.
The above analysis is for graph skeletons, and we also explore the impact of edge labels of two structures in the ABSA task. Figure 2 compares the distribution of edge labels in aspect-opinion paths with the distribution of all edge labels. These distributions are clearly different, both in dependency trees and AMRs, which implies that edge labels can also help the ABSA task, especially in AMRs.
Based on these characteristics, we design the outer product sum module for APARN to mix sentence information into the graph, and design the path aggregator to collect graph skeleton and edge label information in AMRs.
Data-driven Structures Some existing studies use structures produced by data-driven models in the ABSA task (Chen et al., 2020; Dai et al., 2021; Chen et al., 2022) and exhibit different effects from human-defined structures. Therefore, we design a relation-enhanced self-attention mechanism for APARN to integrate the graph information obtained by the path aggregator with the information from the pre-trained model.

heading: 3 Proposed Model
text: The overall architecture of our proposed model APARN is illustrated in Figure 3. It consists of 3 parts: AMR preprocessing, path aggregator and relation-enhanced self-attention mechanism. In the ABSA task, a sentence s = {w1, w2, ..., wn} and a specific aspect term a = {a1, a2, ..., am} are given to determine the corresponding sentiment polarity class ca, where a is a sub-sequence of s and ca ∈ {Positive,Neutral,Negative}.
Many existing works use syntactic dependency trees to establish explicit or implicit connections between aspects and contexts. However, we believe that the sentiment analysis task is essentially about the meanings of sentences, so semantic structures like AMRs are more favorable for this task.
In addition, AMRs are more concise than dependency trees, making it easier to extract valuable information in training but more difficult to preprocess before training. We have to conduct a series of steps including: AMR parsing, AMR aligning and AMR embedding. Preprocessed AMRs still have errors and unsuitable parts for the task, so we design the path aggregator and the relation-enhanced self-attention mechanism to perform joint representation refinement and flexible feature fusion on the AMR graph and the original sentence.
Next, we elaborate on the details of our proposed APARN, including AMR preprocessing and embedding, the path aggregator and the relationenhanced self-attention mechanism.

heading: 3.1 AMR Preprocessing and Embedding
text: Parsing As we determine to employ the semantic structure AMR as an alternative of the syntactic structure dependency tree to better perform the semantic task ABSA, the first step is parsing the AMR from the input sentence. We choose the offthe-shelf parser SPRING (Bevilacqua et al., 2021) for high quality AMR outputs.
Aligning Next, we align the AMR by the aligner LEAMR (Blodgett and Schneider, 2021). Based on the alignments, we manage to rebuild AMR relations between words in the sentence and get the transformed AMR with words as nodes.
Embedding After aligning, we now have transformed AMRs, which can also be called sentences with AMR relations. Then we need to obtain their embeddings for later representation learning by the model. For words in the sentence, also as the nodes in the AMR, we utilize BERT as an encoder to get contextual embeddings H = {h1, h2, ..., hn} like lots of previous works. For the edges in the AMR, we represent the relations between nodes as an adjacency matrix R = {rij | 1 ≤ i, j ≤ n}, where rij is the embedding of the edge label between word wi and word wj . If there is no edge between wi and wj in the AMR, we assign a “none” embedding to rij . Edge label embeddings are also obtained from the pre-trained model.

heading: 3.2 Path Aggregator
text: Path aggregator receives the mix of AMR embeddings R ∈ Rdr×n×n and sentence embeddings H ∈ Rdw×n, where dr and dw denote the dimensions of relation and word embeddings, respectively. Path aggregator outputs the relational fea-
ture matrix RAGG={rAGGij ∈Rdr | 1 ≤ i, j ≤ n}. This process integrates and condenses information from two different sources, AMRs and sentences, making semantic knowledge more apparent but parsing errors less influential.
Outer Product Sum We first add the outer product of two independent linear transformation of sentence embeddings H to the original AMR embeddings R to obtain sequence-enhanced relation embeddings RS ∈ Rdr×n×n. On the one hand, as the outer product of H is the representation of word relations from the sentence perspective, its combination with the AMR embeddings R could enlarge the information base of the model to improve the generalization, also cross validate important features to improve the reliability. On the other hand, AMR embeddings R is usually quite sparse. The outer product sum operation ensures the basic density of the feature matrix and facilitates the subsequent representation learning by avoiding the fuzziness and dilution of numerous background “none” relations to the precious effective relations.
Path Aggregation Next, we perform the path aggregation on RS = {rSij | 1 ≤ i, j ≤ n} to calculate RAGG = {rAGGij | 1 ≤ i, j ≤ n} as:
r′Sij = LayerNorm(r S ij), (1)
ginij , g out ij = sigmoid(Linear(r ′S ij)), (2)
aij , bij = g in ij ⊙ Linear(r′ S ij), (3)
routij = Linear(LayerNorm( ∑
k
aik ⊙ bkj)), (4)
rAGGij = g out ij ⊙ routij . (5)
The path aggregation has distinctive effect on both local and global dissemination of features. From the local view, the path aggregation covers all the 2-hop paths, so that it is very sensitive to neighborhood features, including the features around the aspect term which are really important for the ABSA task. From the global view, information in any long path can be summarized into the representation between the start and the end by several two-in-one operations in enough times of path aggregations. In other words, path aggregations make the features in matrix more inclusive and finally attain global features. In practice, because the ABSA task focuses more on the neighboring information and the BERT encoder with attention mechanisms has made the feature comprehensive enough, a single path aggregation can achieve quite good results.
Additionally, we also introduce a gating mechanism in the path aggregation to alleviate the disturbance of noise from insignificant relations. Finally, the output of path aggregation RAGG is transformed into the relational attention weight matrix AAGG = {aAGGij | 1 ≤ i, j ≤ n} by a linear transformation for subsequent calculation.

heading: 3.3 Relation-Enhanced Self-Attention
text: The classic self-attention (Vaswani et al., 2017) computes the attention weight by this formula:
A = softmax ( QWQ × (KWK)T√
d
) , (6)
where Q and K are input vectors with d dimensions, while WQ and WK are learnable weights
with the same size of Rd×d. In our relation-enhanced self-attention, we added AAGG, the relational attention weight matrix from AMR into the original attention weight, which can be formulated as:
AR=softmax ( HWQ×(HWK)T√
dw +AAGG
) , (7)
where input vectors W and Q are both replaced by the BERT embeddings H with dw dimensions. With AAGG, attention outputs are further guided by the semantic information from AMRs, which improves the efficient attention to semantic keywords.
In addition, similar to path aggregator, we also introduced the gating mechanism into the relationenhanced self-attention as follows:
G = sigmoid(HWG), (8)
HR = (HWV )A R ⊙G, (9)
where WG and WV are trainable parameters and G is the gating matrix. Considering the small proportion of effective words in the whole sentence, the gating mechanism is conducive to eliminating background noise, making it easier for the model to focus on the more critical words.
Finally, with all these above calculations including relation-enhanced self-attention and gating mechanism, we obtain the relation-enhanced aspect representation HRa = {hRa1 , hRa2 , ..., hRam} for subsequent classification.

heading: 3.4 Model Training
text: The final classification features are concatenated by the original BERT aspect representation Ha = mean{ha1 , ha2 , ..., ham} and the relationenhanced aspect representation HRa .
Hfinala = [Ha, H R a ]. (10)
It is passed through a fully connected softmax layer and mapped to probabilities over three sentiment polarities.
p(a) = softmax(WpH final a + bp). (11)
We use cross-entropy loss as our objective function:
LCE = − ∑
(s,a)∈D
∑ c∈C yca log p c(a), (12)
where y is the ground truth sentiment polarity, D contains all sentence-aspect pairs and C contains all sentiment polarities.

heading: 4 Experiments
text: In this section, we first introduce the relevant settings of the experiments, including the datasets used, implementation details and baseline methods for comparison. Then, we report the experimental results under basic and advanced settings. Finally, we select several representative examples for model analysis and discussion.

heading: 4.1 Datasets and Setup
text: Our experiments are conducted on four commonly used public standard datasets. The Twitter dataset is a collection of tweets built by Dong et al. (2014), while the Restaurant and Laptop dataset come from the SemEval 2014 Task (Pontiki et al., 2014). MAMS is a large-scale multi-aspect dataset provided by Jiang et al. (2019). Data statistics are shown in Appendix A.1.
In data preprocessing, we use SPRING (Bevilacqua et al., 2021) as the parser and LEAMR (Blodgett and Schneider, 2021) as the aligner. APARN uses the BERT of bert-base-uncased version with max length as 100 and the relation-enhanced selfattention mechanism uses 8 attention heads. We reported accuracy and Macro-F1 as results which
are the average of three runs with different random seeds. See Appendix A.2 for more details.

heading: 4.2 Baseline Methods
text: We compare APARN with a series of baselines and state-of-the-art alternatives, including: 1) BERT (Devlin et al., 2019) is composed of a general pre-trained BERT model and a classification layer adapted to the ABSA task. 2) DGEDT (Tang et al., 2020) proposes a dual transformer structure based on dependency graph augmentation, which can simultaneously fuse representations of sequences and graphs. 3) R-GAT (Wang et al., 2020) proposes a dependency structure adjusted for aspects and uses a relational GAT to encode this structure. 4) T-GCN (Tian et al., 2021) proposes an approach to explicitly utilize dependency types for ABSA with type-aware GCNs. 5) DualGCN (Li et al., 2021) proposes a dual GCN structure and regularization methods to merge features from sentences and dependency trees. 6) dotGCN (Chen et al., 2022) proposes an aspectspecific and language-agnostic discrete latent tree as an alternative structure to dependency trees. 7) SSEGCN (Zhang et al., 2022b) proposes an aspect-aware attention mechanism to enhance the node representations with GCN.

heading: 4.3 Main Results
text: Table 2 shows the experimental results of our model and the baseline models on four datasets under the same conventional settings as Li et al. (2021), where the best results are in bold and the second best results are underlined. Our APARN exhibits excellent results and achieves the best results on all 8 indicators of 4 datasets with an average margin more than one percent, which fully proves the effectiveness of this model.
Comparing the results of different datasets, we can find that the improvement of APARN on the Twitter dataset is particularly obvious. Compared to the best baselines, the accuracy rate has increased by 1.65% and the Macro-F1 has increased by 1.79%. The main reason is the similarity of the Twitter dataset to the AMR 3.0 dataset, the training dataset for the AMR parser we used. More than half of the corpus of the AMR 3.0 dataset comes from internet forums and blogs, which are similar to the Twitter dataset as they are both social media. As a result, the AMR parser has better output on the Twitter dataset, which in turn enables the model to
extract more valuable features from it and leads to a considerable improvement. This difference among datasets also reflects the effectiveness of semantic information from AMR for the ABSA task.

heading: 4.4 Comparative Experiments
text: We conduct comparative experiments to analyse the impact of models (APARN and T-GCN), parsed structures (AMR and dependency tree), and edge labels (with and without). T-GCN is selected instead of more recent models because they lack the ability to exploit edge labels and cannot receive AMRs as input. AMRs are the same as the basic experiments and dependency trees are parsed by Stanford CoreNLP Toolkits (Manning et al., 2014). “Without edge labels” means all labels are the same placeholder. The results are shown in Figure 4.
From the perspective of models, APARN consistently outperforms T-GCN in any parsed structure and edge label settings, demonstrating the effectiveness of our APARN. From the perspective of parsed structures, AMRs outperform dependency trees in most model and edge label settings, except for the case of T-GCN without edge labels. The reason may be that the AMR without edge labels is sparse and semantically ambiguous, which does not match the design of the model.
From the perspective of edge labels, a graph with edge labels is always better than a graph without edge labels, whether it is an AMR or a dependency tree, whichever the model is. We can also notice that APARN has a greater improvement with the addition of edge labels, indicating that it can utilize edge labels more effectively. Besides, with the addition of edge labels, experiments using AMR have improved more than experiments using depen-
dency trees, indicating that edge labels of the AMR contain richer semantic information and are more valuable for sentiment analysis, which is consistent with previous experiments in Figure 2.

heading: 4.5 Further Analysis
text: Ablation Study To analyze the role of each module, we separately remove four key components of APARN. Results on four datasets are represented in Table 3.
According to the results, each of the four components contributes significantly to the performance of APARN. Removing Outer Product Sum results in a significant drop in performance, illustrating the importance of promoting consistency of information from sentences and AMRs. Removing Path Aggregator is worse than removing Relation in SelfAttention, indicating that unprocessed AMR information can only interfere with the model instead of being exploited by the model.
Comparing the results in different datasets, we can find that the model depends on information from sentences and AMRs differently on different datasets. On the Restaurant dataset, removing the Relation in Self-Attention component has less impact, while on the Twitter dataset, removing this component has a greater impact. This means the model utilizes sentence information more on the Restaurant dataset and AMR information more on the Twitter dataset. This is also consistent with the analysis of the main results: the AMR of Twitter dataset has higher quality due to the domain relatedness with the training dataset of the AMR parser, which in turn makes the model pay more attention to the information from the AMR on this dataset.
AMR Parser Analysis We conduct experiments using AMRs from different parsers on Twitter dataset, as displayed in Figure 5. In addition to the SPRING parser mentioned before, we try two other parsers from Zhang et al. (2019b) and Cai and Lam (2020). These parsers achieve 76.3, 80.2
and 84.3 Smatch score for AMR parsing task on AMR 2.0 dataset, which can be regarded as the quality of their output. From the figure, it is clear that the accuracy of ABSA task shows positive correlation with the Smatch score, which proves the positive effect of AMRs in the ABSA task and the importance of the high quality AMR.
Sentence Length Study Table 4 compares the accuracy of APARN with and without path aggregator for sentences of different lengths in the Restaurant dataset. According to the table, we can see that the model achieves higher accuracy on short sentences, while the long sentences are more challenging. In addition, the model with the path aggregator has a larger relative improvement on long sentences than short sentences, indicating that the path aggregator can effectively help the model capture long-distance relations with AMR.

heading: 4.6 Case Study
text: As shown in Figure 6, we selected three typical cases to visualize the aspect terms’ attention to the context before and after adding information from the AMR, respectively.
From the first two examples, we can notice that the model focuses on the copula verb next to the opinion term without the AMR. While with the information from the AMR, the model can capture opinion terms through the attention mechanism more accurately. In the third example, without the AMR, the model pays more attention to words that are closer to the aspect term. With the semantic information from AMR, the model can discover opinion terms farther away from aspect terms.
These cases illustrate that the semantic structure information of AMR plays an important role in making the model focus on the correct opinion words. It also shows that the structure of our APARN can effectively utilize the semantic structure information in AMR to improve the performance in the ABSA task.

heading: 5 Related Work
text: Aspect-based Sentiment Analysis Traditional sentiment analysis tasks are usually sentence-level or document-level, while the ABSA task is an entity-level and fine-grained sentiment analysis task. Early methods (Jiang et al., 2011; Kiritchenko et al., 2014) are mostly based on artificially constructed features, which are difficult to effectively model the relations between aspect terms and its context. With the development of deep neural networks, many recent works (Wang et al., 2016; Tang et al., 2016; Chen et al., 2017; Fan et al., 2018; Gu et al., 2018; Du et al., 2019; Liang et al., 2019; Xing et al., 2019) have explored applying attention mechanisms to implicitly model the semantic relations of aspect terms and identify the key opinion
terms in the context. Another trend in ABSA studies is the explicit use of dependency trees. Some works (He et al., 2018; Zhang et al., 2019a; Sun et al., 2019; Huang and Carley, 2019; Zhang and Qian, 2020; Chen et al., 2020; Liang et al., 2020; Wang et al., 2020; Tang et al., 2020; Phan and Ogunbona, 2020; Li et al., 2021; Xiao et al., 2021) extend GCN, GAT, and Transformer backbones to process syntactic dependency trees and develop several outstanding models. These models shorten the distance between aspect terms and opinion terms by dependency trees and alleviate the long-term dependency problem.
Recent studies have also noticed the limitations of dependency trees in the ABSA task. Wang et al. (2020) proposes the reshaped dependency tree for the ABSA task. Chen et al. (2020) propose to combine dependency trees with induced aspect-specific latent maps. Chen et al. (2022) further proposed an aspect-specific and language-independent discrete latent tree model as an alternative structure for dependency trees. Our work is similar in that we also aim at the mismatch between dependency trees and the ABSA task, but different in that we introduce a semantic structure AMR instead of induced trees.
Abstract Meaning Representation AMR is a structured semantic representation that represents the semantics of sentences as a rooted, directed, acyclic graph with labels on nodes and edges. AMR is proposed by Banarescu et al. (2013) to provide a specification for sentence-level comprehensive semantic annotation and analysis tasks. Research on AMR can be divided into two categories, AMR parsing (Cai and Lam, 2020; Zhou et al., 2021; Hoang et al., 2021) and AMR-to-Text (Zhao et al., 2020; Bai et al., 2020; Ribeiro et al., 2021).
AMR has also been applied in many NLP tasks. Kapanipathi et al. (2020) use AMR in question answering system. Lim et al. (2020) employ AMR
to improve common sense reasoning. Wang et al. (2021) utilize AMR to add pseudo labels to unlabeled data in low-resource event extraction task. Our model also improves the performance of the ABSA task with AMR. Moreover, AMR also has the potential to be applied to a broader range of NLP tasks, including relation extraction(Hu et al., 2020, 2021a,b), named entity recognition(Yang et al., 2023), natural language inference(Li et al., 2022), text-to-SQL(Liu et al., 2022), and more.

heading: 6 Conclusion
text: In this paper, we propose APARN, AMR-based Path Aggregation Relational Network for ABSA. Different from the traditional ABSA model utilizing the syntactic structure like dependency tree, our model employs the semantic structure called Abstract Meaning Representation which is more harmony with the sentiment analysis task. We propose the path aggregator and the relation-enhanced selfattention mechanism to efficiently exploit AMRs and integrate information from AMRs and input sentences. These designs enable our model to achieve better results than existing models. Experiments on four public datasets show that APARN outperforms competing baselines.

heading: 7 Limitations
text: The high computational complexity is one of the biggest disadvantages of the path aggregation. The time consumption and GPU memory used for multiple operations are expensive. So it is very desirable to use only one time of path aggregation due to attributes of the ABSA task in our APARN.
Another limitation of this work is that the performance of the model is still somewhat affected by the quality of the AMR parsing results. The good news is that the research on AMR parsing is continuing to make progress. In the future, APARN with higher quality AMRs is expected to further improve the level of the ABSA task.
Besides, this model is flawed in dealing with implicit and ambiguous sentiments in sentences. Implicit sentiment lacks corresponding opinion words, and ambiguous sentiment is subtle and not apparent. An example of this is the sentence "There was only one [waiter] for the whole restaurant upstairs," which has an ambiguous sentiment associated with the aspect word "waiter". The golden label is "Neutral", but our model predicts it as "Negative".
Finally, generalization to other ABSA tasks such
as end-to-end ABSA or ASTE is another restriction. Considering the complexity of the task, we only apply our motivation to sentiment classification in this paper. We will further generalize it to more complex sentiment analysis tasks in the future work.

heading: Acknowledgements
text: The work was supported by the National Key Research and Development Program of China (No. 2019YFB1704003), the National Nature Science Foundation of China (No. 62021002), Tsinghua BNRist and Beijing Key Laboratory of Industrial Bigdata System and Application.

heading: A Appendix
text: A.1 Datasets The statistics for the Restaurant dataset, Laptop dataset, Twitter dataset and MAMS dataset are shown in Table 5. Each sentence in these datasets is annotated with aspect terms and corresponding polarities. Following Li et al. (2021), we remove instances with the “conflict” label. So all datasets have three sentiment polarities: positive, negative and neutral. Throughout the research, we follow the Creative Commons Attribution 4.0 International Licence of the datasets.
A.2 Implementation Details Preprocessing We use SPRING (Bevilacqua et al., 2021) as the parser to obtain the AMRs of input sentences and use LEAMR (Blodgett and Schneider, 2021) as the AMR aligner to establish the correspondence between the AMRs and sentences. The maximum length of the input sentence
is set to 100, the shortage is made up with the special word “PAD” and the excess is truncated.
Some edge labels are treated specially when mapping the edges of AMR to the relations between words. Edge labels suffixed with “-of” are used to avoid loops in AMR, so we swap their start and end points and remove the “-of” suffix, eg: the “:ARG0-of” relation from tokeni to tokenj is changed to the “:ARG0” relation from tokenj to tokeni. Edge labels prefixed with “:prep-” are used because there is no suitable preposition label in the AMR specification. We changed them to original prepositions, for example, “:prep-against” is changed to “against”.
Model Structure and Training APARN uses the BERT of bert-base-uncased version as a pretrained encoder. The dimension of its output is 768, which is also used as the dimension of token representation in the path aggregator. The dimension of the AMR edge label embedding derived from the SPRING model is 1024. Due to computational efficiency and memory usage, this dimension is reduced to 376 through a linear layer as the dimension of the relational matrix features in the path aggregator. For the relation-enhanced selfattention mechanism, its gated multi-head attention mechanism uses 8 attention heads with the latent dimension size of 64. The total parameter size of APARN is about 130M and it takes about 8 minutes to train each epoch on a single RTX 3090 GPU with the batch size of 16.
During training, we use the Adam (Kingma and Ba, 2015) optimizer and use the grid search to find best hyper-parameters. The range of learning rate is [1 × 10−5, 5 × 10−5]. Adam hyperparameter α is 0.9 and β is in (0.98, 0.99, 0.999). The BERT encoder and other parts of the model use dropout strategies with probability in [0.1, 0.5], respectively.
Each training lasts up to 15 epochs and the model is evaluated on validation data. For datasets without official validation data, we follow the settings of previous work (Li et al., 2021). The model with the highest accuracy among all evaluation results is selected as the final model.
A.3 More Comparison Examples
Here are two other comparison examples of dependency trees (Figure 7) and AMRs (Figure 8).
The first sentence is “We usually just get some of the dinner specials and they are very reason-
ably priced and very tasty”. In its dependency tree, the distance between the aspect “dinner specails” and the opinion terms “reasonably priced” or “very tasty” is more than 3, while they are directly connected in the AMR.
The second sentence is “We parked on the block of Nina ’s the place looked nice , with people obviously enjoying their pizzas”. In its dependency tree, the distance between the aspect “place” and the opinion terms “nice” is 4, while they are directly connected in the AMR.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Limitations
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: 3.1 and 4.1 and 4.5

heading: 3 B1. Did you cite the creators of artifacts you used?
text: 3.1 and 4.1 and 4.5

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: A.1
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? A.1
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. A.1
C 3 Did you run computational experiments? 4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? A.2
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 4.1 and A.2

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4 and A.2

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? A.2
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: In the future, APARN with higher quality AMRs is expected to further improve the level of the ABSA task. Besides, this model is flawed in dealing with implicit and ambiguous sentiments in sentences. Implicit sentiment lacks corresponding opinion words, and ambiguous sentiment is subtle and not apparent. An example of this is the sentence "There was only one [waiter] for the whole restaurant upstairs," which has an ambiguous sentiment associated with the aspect word "waiter". The golden label is "Neutral", but our model predicts it as "Negative". Finally, generalization to other ABSA tasks such
as end-to-end ABSA or ASTE is another restriction. Considering the complexity of the task, we only apply our motivation to sentiment classification in this paper. We will further generalize it to more complex sentiment analysis tasks in the future work