Tagging-Assisted Generation Model with Encoder and Decoder Supervision for Aspect Sentiment Triplet Extraction
Authors: Luo Xianlong
Meng Yang
Yihao Wang
Venue: EMNLP 2023 Main (2023)
Keywords: ABSA; ASTE; sentiment analysis; generation seq2seq model; sequence tagging;  semantic alignment
TL;DR: We propose the TAGS model that leverages sequence tagging task and label semantic representations simultaneously to enhances the supervision of encoder and decoder hidden states

Abstract: ASTE (Aspect Sentiment Triplet Extraction) has gained increasing attention. Recent advancements in the ASTE task have been primarily driven by Natural Language Generation-based (NLG) approaches. However, most NLG methods overlook the supervision of the encoder-decoder hidden representations and fail to fully utilize the semantic information provided by the labels to enhance supervision. These limitations can hinder the extraction of implicit aspects and opinions. To address these challenges, we propose a tagging-assisted generation model with encoder and decoder supervision (TAGS), which enhances the supervision of the encoder and decoder through multiple-perspective tagging assistance and label semantic representations.
    Specifically, TAGS enhances the generation task by integrating an additional sequence tagging task, which improves the encoder's capability to distinguish the words of triplets. Moreover, it utilizes sequence tagging probabilities to guide the decoder, improving the generated content's quality.
    Furthermore, TAGS employs a self-decoding process for labels to acquire the semantic representations of the labels and aligns the decoder's hidden states with these semantic representations, thereby achieving enhanced semantic supervision for the decoder's hidden states. Extensive experiments on various public benchmarks demonstrate that TAGS achieves state-of-the-art performance.

Tagging-Assisted Generation Model with Encoder and Decoder Supervision for Aspect Sentiment Triplet Extraction
Xianlong Luo Meng Yang Yihao Wang
"Scooo of Computer Science and Engineering, Sun Yat-Sen University 2 Key Laboratory of Machine Intelligence and Advanced Computing (SYSU), Ministry of Education, China
luoxlong mail2.sysuu.du..cn yangmb mail. sysu.edu.cn, wangyh357 wts.7mmmail..ssseduuuu..
Abstract
the explanation for sentiments, explicitly highlight- ing the causes of sentiments and the entities to which they are attached. This task involves address- ing challenges such as the diversity of emotional expressions and the complexity of linguistic con- texts. For instance, in the sentence "Food wise, it's ok but a bit pricey for what you get considering the restaurant isn't an'ta fancy place," " three sentiment triplets can be extracted: (food, ok, neutral), (food, pricey, neutral), and (restaurant, isn't 't a fancy place, neutral).
ASTE (Aspect Sentiment Triplet Extraction) has gained increasing attention. Recent ad- vancements in the ASTE task have heen pri- marily driven by Natural Language Generation- based (NLG) approaches. However, most NLG methods overlook the supervision of the encoder-decoder hidden representations and fail to fully utilize the semantic information provided by the labels to enhance supervision. These limitations can hinder the extraction of implicit aspects and opinions. To address these challenges, we propose a tagging-assister gen- eration model with encoder and decoder super- vision (TAGS), which enhances the supervision the encoder and decoder through multiple- perspective tagging assistance and label se- mantic representations. Specifically, TAGS enhances the generation task by integrating additional sequence tagging task, which improves the encoder's capability to distin- guish the words of triplets. Moreover, uti- lizes sequence tagging probabilities to guide the decoder, improving the generated content's quality. Furthermore, TAGS employs self- decoding process for labels to acquire the se- mantic representations of the labels and aligns the decoder's hidden states with these seman- tic representations, thereby achieving enhanced semantic supervision for the decoder's hidden states. Extensive experiments various public benchmarks demonstrate that TAGS achieves sfate-of-hhe-r performance.
Existing Methods The current mainstream ap- proaches for ASTE can be classified into two cat- egories: sequence tagging- basee approaches and sequence generation-based approaches. ASTE em- ployed a sequence tagging method initially intro- duced by Peng et al. (2020). However, the sequence lagging-base approaches in ASTE fail to capture the semantic information conveyed by the labels, which can result in semantic mismatches in the pre- dicted results (Zhang al., 202 By leveraging the rich label semantic information and mitigating the potential error propagation pipeline methods (Paolini et al., 2021; Yu et al., 2023), generation methods achieve better performance in ASTE.
Geenration-baseee approaches still face two sig- nificant challenges. Firstly, the supervision of hid- den representations within encoder-decoder archi- tectures has been overlooked, leading to potential issues such as the degeneration of neural language models and difficulty in identifying distinctive in- formation (Su et al.. 2022). In the context of the ASTE task, this oversight plicit aspects and opinions (Cai et al., 2021; Peper and Wang, 2022). Secondly, during training, the se- mantic information of the labels has yet to fully utilized. Traditional supervision utilizes labels the form of one-hot probability vectors without fully leveraging the semantic information of the labels at the hidden state level.
1 Introduction
Aspect Sentiment Triplet Extraction (ASTE) aims to extract sentiment triplets from a sentence, i.e., Aspect: the aspect term represents an explicit men- tion of a discussed target, Opinion: the mentioned comment terms/phrases, Sentiment: sentiment po- larity of the aspect, holding significant potential in downstream research and applications. Unlike sen- tence sentiment classification, ASTE emphasizes
fail to extract im-
*Corresponding author.
TAGS To address the challenges mentioned

---

above, we propose novel tagging-assisted gen- eration model called TAGS, which enhances the su- pervision of both the encoder and decoder through tagging assistance and label semantic representations. TAGS consists of two modules: "Empowering Generation through Se- quence Tagging" (EGST) and "Label-Driven Se- mantic Alignment" (LDSA).
bels the decoder level and achieve semantic alignment of the decoder's hidden state to the labels in the generation model.
3. The experimental results show that pro- posed framework significantly outperforms recent SOTA methods.
Problem statement
In EGST, utilize sequence tagging task enhance the generation task through three as- pects: Multitask Learning, Guided Generation, and Result Optimization. Multitask learning: we en- hance the supervision in the encoder of the gen- eration model by introducing a sequence tagging task. This additional task empowers the encoder to distinguish between triplet and irrelevant words effectively, thereby benefiting the generation task. Guided Generation: We incorporate the sequence tagging outputs into the decoder's attention mech- anism. This encourages the model to focus more on the keywords identified by the sequence tag- ging task. Result Optimization: Finally, during inference, we utilize the sequence tagging results to optimize the generation results, thereby improving the quality of the results.
The input of the ASTE task is a sentence X {I 22. Fn where each represents word and n is the maximum length of the sentence. The goal of the ASTE task is generate set of sen- timent triplets
= {(a, o, $)&) k=1 where [T] means the number of triplets in T. Each triplet consists of an aspect term (a), an opinion term (o), and the corresponding sentiment polarity (s) (s (POS, FU, N F.G)). ).
Our proposed TAGS an encoder-decoder model designed for the generation task, in which the input a natural sentence and the generation target, i.e., the label sentence, constructed by concatenating triplets from the set T as follows:
Y = "a1, 01, 02; 02; : ag 0k, 8&", where 0g Oj, and 8; correspond to the i-th triplet (a, o, 8);.
In LDSA, we further enhance the supervision for the decoder's hidden states in the generation model by utilizing the semantic information conveyed by labels. Firstly, we convert label triplets into a nat- ural context, referred to as a label sentence, and input the label sentence into the TAGS model to obtain more accurate hidden state, which also serves as a semantic label representation. Subse- quently, dynamically align the hidden states of the decoder to the label's semantic representation according to the comparison results between the tokens corresponding to the semantic representa- tion and the ground truth tokens. By this alignment, the model can better capture the semantic informa- tion conveyed by the labels, making the generation more in line with the intended label semantics.
3 Methodology
shows our proposed TAGS method. TAGS comprises two modules, an Empowering Generation through Sequential Tagging module (EGST) and a Label-Driven Semantic Alignment (LDSA) module. EGST leverages sequence tag- ging task to enhance the generation model in three aspects: Multitask Learning, Guided Generation, and Result Optimization. LDSA utilizes label self-decoding process obtain the semantic repre- sentation of labels and aligns the decoder's hidden states to the semantic representation during train- ing, thereby achicving enhanced semantic supervi- sion for the decoder's hidden states.
Fig.
3.1 Empowering Generation through Sequence Tagging
Extensive experimental results validate the ef- fectiveness of the TAGS model. In summary, our contributions this work are threefold:
TAGS leverages sequence tagging to enhance the generation task from multiple perspectives, shown in the right part of Fig. 1. Firstly, TAGS employs sequence tagging task as an additional task to enhance the supervision of the encoder, thereby improving its ability to differentiate between triplet and irrelevant words. By sharing parameters be- tween the sequence tagging model and the gener- ation model, the enhanced discriminative power
1. We propose a novel ASTE generation model, which utilizes sequence tagging to assist the generation via enhancing the supervision of the encoder's hidden state and incorporating sequence tagging probabilities and results to improve the generation process.
2. We obtain the semantic representation of la-

---

Empowering Generation through Sequential Labeling
Final Result
Aligament
Lggneeraioon
compare
Infereace
Decoder
"aligament
Y'
y'a
Leagging
$
Lm head
21. papill prngg)
Similarity
Linear
Σ a dhe
%
%
d
1
h>
BA,
Encuder& Decoder
shuring weight
Decoder
Encoder
X₂
Y1
Y
Inpor
Labell Sentence
Enput
Figure I: Overview of our TAGS framework, which consists of two parts: Empowering Generation through Sequence Tagging (right) and Label-Driven Semantic Alignment (left). Sequence tagging enhances the generation task in three aspects. Encoder Multitask Learning: Liagging for sequence tagging task. Guided Generation: the decoder's s Tag Attention incorporates probabilities (p;) from the sequence tagging task as additional weights for its cross-attention mechanism. (The figure illustrates the generation of the q-th word.) Result Optimization: "Inference" optimizes the generation results using tagging results. Semantic representation of the label (h),) is obtained by inputting the label sentence (Y) into the model. Lalignment is computed based on the cosine similarity results (h and h and the alignment labels L.
obtained from the sequence tagging task can also benefit the triplets extraction process in the genera- tion model. Next, the sequence tagging task prob- abilities are integrated into the generation model, compelling it to prioritize the words identified as crucial by the sequence tagging results. This inte- gration ensures that the generation model produces content closely aligned with those words. Lastly, TAGS utilizes the sequence tagging task results dur- ing inference optimize the generated results. By considering the results from both methods, TAGS achieves more comprehensive information fusion, enhancing overall model performance.
noted as (X, Z), where Z is the tagging label {21, 22, 23...., 2n). The encoder encodes obtain hidden states H.Y
H En([x1.22
.., .n)) 2n))
(1)
where En is Encoder, H E Raxd d denotes the hidden dimension. H is also the encoder hidden state for the generation task. Pass H through a fully connected layer to obtain the tag probabilities pi:
Pj softmax Senttmax(WMttt (Wi bb)
(2)
where W1 7xd by E RR. and p; RRT repre- sents the probability distribution of the i-th word across tags. We calculate sequence tagging loss using cross-entropy loss:
3.1.1 Sequence Tagging Task
We perform multitask learning by simultaneously training generation task and sequence tagging task.
Tagging Scheme In our designed sequence tag- ging scheme, each word will be classified into one of categories. The "N" category represents non- keywords, while the remaining 6 categories rep- resent aspects and opinions, each combined with three sentiment types (positive, negative, and neu- tral). Read Appendix A.1 for detailed descriptions. Tagging Task tagging sample is de-
Laggiing
CE(P, 24)
(3)
3.1.2 Generation guided by sequence tagging Tag Attention To leverage the guidance informa- tion the sequence tagging task provides, we com- pute the probability i the i-th word being a keyword. po[ denotes the probability of the i-th

---

A Alignment Labels The main objective of seman- tic alignment is to establish alignment between H and H One significant challenge arises from the fact that even though HD. represents a more accu- rate hidden state, its corresponding output tokens Y', as shown in Equation 7, may not always match the ground truth token sequence Y during the early stuges of training. Therefore, compare y with y/, and only when v/ equal yrs it indicates that ha correct. We then allow he) to be close hy word belonging the non-keyword category ("N"), Consequently, pi 1 [.] indicates the prob- ability of the i-th word belonging to the keyword category. We incorporate i into the cross- attss-nttentoon mechanism of the decoder in the generation model as follows:
training, We align the decoder's hidden state the semantic representation, thereby enhancing the supervision of the decoder's hidden state. This alignment ensures that the generated output closely matches the semantic content of the label.
Label Semantic Representation During train- ing, we input the label sentence Y into the model to obtain the decoder's hidden state:
exp((1 + pc) as) ON Σj=1 ( + Pÿ) onj)
(4)
H En-De((nn U2.... Mn.) [M ...., AM (6)
where ou represents the attention score at the t- th row and i-th column in the cross-attention score matrix before applying softmax. ān denotes the fi- nal adjusted attention score after applying softmax. (1 Pi) ensures balanced contribution from both the sequence tagging task and inherent generation task to the attention distribution. Compared to the formulation without adding 1 to i. (Appendix A.3), this formulation effectively enhances the genera- tion process while mitigating the potential impact of tagging errors on overall generation quality (Ap- pendix B.2).
where En-De means mncoder-decoder architec- ture. Since the label sentence contains only the words of the correct triplets, the model can effort- lessly extract the correct triplets from it. In this case, the model's input and output are both the la- bel sentence, essentially forming a self-decoding process. Furthermore, due to the absence of ir- relevant words the input, HD is more accu- De
rate compared to H where H De( - [ h ..... A as demonstrated in Experiment 4.3.3. Therefore, we regard H as accurate se- mantic representation of the label that can provide substantial supervision at the decoder stage.
Generation Task A sample is denoted as (X, Y), where Y represents the label sentence {3n, 3/2 Vs...., }m with m being the maximum length of Y. The loss function for the generation task with model parameters 0 is defined as follows:
C
generattin =
log lo
(5)
3.1.3 Inference
During inference, we leverage the sequence tag- ging results to optimize the generated outputs. The main operation involves comparing the generated triplets from the generation task with the triplets from the sequence tagging task. If the generated aspect is subset of the sequence tagging aspect set, or vice versa, and the generated opinion is a subset of the sequence tagging opinion set, or vice versa, the triplet retained. Otherwise, the triplet is discarded. Read Appendix A.2 for details.
Otherwise, we move hg away from N. Use L to represent the comparison result between yf and y::
Y" (Lm_head (H)))) argmmax)
(7)
L = Equal 3:)
(8)
where Lm head represents linear layer that takes the decoder's hidden states as input and outputs a probability distribution over the vocabulary. The predicted tokens Y' are obtained by selecting the words with the highest probability using the argmaz operation. The function "Equal" outputs I when the inputs equal and 0 otherwise.
3.2 Label-Driven Semantic Alignment
The form of supervision, similar to Equation 5, lacks fine-grane fine-grained supervision at the hidden state level and fails to fully utilize the semantic informa- tion embedded in the labels. In Label-Driven Se- mantic Alignment (shown in the left part of Fig. 1), we employ a label self-decoding process to obtain a more accurate decoder hidden state, which serves as a semantic representation of the label. During
Alignment Task Alignment is achieved by ad- justing the distance between h.x and by accoding to L, Employ cosine similarity to quantify the distance:
8; cos( hi R
(9)
s Relu(ss)
(10)

---

where cos is the cosine similarity, and ReLu is used limit the similarity values between 0 and (Appendix B.3). We compute the alignment loss using binary cross-entropy to enforce the cosine similarity scores to align with the labels L:
Table 1: Statistics of datasets. and mean the total number of sentences and triplets. POS, NEU, and NEG represent the number of positive, neutral, and negative sentiment triplets, respectively.
Datasct
$
T
POS
NEU
NEG
train
605
1013
783
25
205
15Rcs
Lalignment
Σ BCEloss($,, I.))
(11)
dev
148
249
185
11
53
test
322
485
317
25
143
train
857
1394
1015
50
329
Final Loss. Therefore, the final loss defined as follows:
16Res
dev
210
339
252
11
76
326
514
407
29
78
L = Cgnnrrrctioonn + 0r Ciaging + 03 Calignment
14Lap
train
906
1460
817
126
517
(12)
219
345
169
36
140
test
328
541
364
63
114
where a1, a2 and ag are hyperparameters.
train
1266
2337
1015
50
329
14Rcs
dev
310
577
252
11
76
4
Experiments
TesE
492
994
407
29
78
4.1
Experiment Setup
ASTE Dataset We evaluate our TAGS on four popular ASTE datasets shown in Table 1: 14Res, 14Lap, 15Res, 16Res (Pontiki et al., 2014, 2015, 2016), which are modified for ASTE task by Fan et al. (2019); Peng al. (2020); Xu et al. (2020a); Wu al. (2020).
linear layers. The model is trained for 40 epochs Nvidia 3090 GPUs, and the hyperparameters of Equation 12 are set as follows: 01 10, a2 1, and 03 = The probability threshold in the infer- ence stage is 0.999. All the reported results are the average of five runs with different random seeds.
Baseline Models We categorize the comparison
Evaluation Metrics Following previous works (Peng et al.. 2020), we employ widely used eval- uation metrics, namely F1 scores (F1), recall (R), and precision (P).
models into the following three types:
1.Sequence tagging-based models, such as OTE-
MTL (Zhang et al., 2020), GTS (Wu et al., 2020), JET (Xu et al., 2020b), EMC-GCN (Chen et al., 2022a), SyMux (Fei et al., 2022), SCEDD (Zhang al., 2022b), BDTF (Zhang al., 2022a), SA- Transformer (Yuan et al., 2023), STAGE (Liang et al., 2023).
4.2 Main Results
The main results reported in Table 2. In this task, Fi is the most important metric (Peng et al., 2020; Chen et al., 2022b; Gao et al., 2022; Gou et al., 2023). TAGS significantly outperforms the previous state-of-the-are method Mvp (Gou et al., 2023), specifically achieving a lead of up to 3.13% on the 16res dataset and 2.01% % on the 15res dataset according to the F1 metric.
2.Generation-based models, such as GAS(Zhang et al.. 2020), Paraphrase (Wu et al., 2020), BARTABSA (Yan et al., 2021), PASTE (Mukher- jee et al., 2021), Seq2Path (Mao et al., 2022), DLO (Hu et al., 2022a), LEGO-ABSA (Gao et al., 2022), EHG (L.v al., 2023) and Mvp (Gou al., 2023). 3.Models based on other methods: reinforee- ment learning based model ASTE-RL (Jian et al., 2021), reading comprehension based model BMRC (Chen et al., 2021), and span-level models Span-ASTE (Xu et al., 2021) )) and SBN (Chen et al., 2022b).
Based on the principles of sequence tagging- based methods, these approaches tend to be con- servative, which means they only predict a triplet when they are highly confident. Consequently, the precision of these methods tends to be higher than the recall, as shown in both the OTE-MTL and JET methods in Table 2. In contrast, generation meth- ods tend to over-predict the number of triplets due to their strong creativity. Consequently, the recall in the results of generation methods is generally higher than the precision.
Experiment Details We employ the T5-base model (Raffel et al., 2020) from the huggingface Transformer library as our pre-trained generative encoder-decoder decoder model. During training, we set the learning rate to 3c-4 for T5 and 5e-3 for all the
By introducing a sequence tagging task, the

---

Table 2: Main results on datasets of ASTE tasks. The best results are in bold, while the second best are underlined. denotes the replication results, while the other results are obtained from original papers.
16res
15res
14lap
14res
Model
P
R
P
R
P
F1
F1
BMRC
71.20
61.08
65.75
68.51
53.40
60.02
70.55
48.98
57.82
75.61
61.77
67.99
Other
ASTE-RL
67.21
69.69
68.41
65.45
60.29
62.72
64.80
59.50
Span-ASTE
69.45
71.17
70.26
62.18
64.45
63.27
63.44
55.84
59.38
72.89
70.89
71.85
SBN
71,59
72:57
72.08
69.93
60.41
64.82
65.68
59.88
62.65
76,36
72.43
74.34
OTE-MTL
62.88
52.10
56.96
56.37
40.94
47.13
49.53
39.22
43.42
62.00
55.97
58.71
JET
70.42
58.37
63.83
64,45
51.96
57.53
55.39
47.33
$1.04
70.56
55,94
62.40
GTS
66.08
69.91
67.93
62.59
57.94
60.15
57.82
$1.32
$4.36
67.76
67.29
67.50
Tagging
EMC-GAN
64.43
72.63
67.69
60.45
62.72
61.55
59.61
56.30
57.90
70.37
72.84
71.58
SyMux
\
72.76
63.13
60.11
74.84
SCEDD
66.11
71.37
68.64
59.41
62.73
61.03
61.84
60.08
60.95
70.27
73.02
71.62
Terrransformer
72.01
62.87
67.13
62.82
58.31
60.48
61.28
48.98
54.44
70.76
65.85
68.22
BDTF
71.44
73.13
72.27
68.76
63.71
66.12
68.94
55.97
61.74
75.53
73.24
74.35
STAGE
77.67
68.44
72.75
72.33
58.93
64.94
70.56
55.16
61.88
78.51
69.30
73.61
GAS
70.10
62.10
60.78
72.16
Paraphrase
\
71.70
62.56
61.13
72.03
BARTASA
66.6
68.68
67.62
59.14
59.38
59.26
61.41
56.19
65.52
65.25
Generation
PASTE
66.1
69.8
67.9
61.7
60.8
61.3
61.2
53.6
$7.1
66.7
66.5
66.6
DLO
72.23
63.52
61.33
72.02
Soq2patht
71.59
75.41
73.40
62.62
65.48
64.02
64.57
62.22
73.28
73.75
LEGO-ABSA
69.9
64.4
62.2
73.7
EHG
72.35
63.58
61.53
71.82
MvP
73.48
65.89
63.33
74.05
TAGS
76.37
76.85
76.61
70.23
65.73
67.90
65.
62.20
64.53
77.38
72.86
75.05
TAGS method alleviates the excessive creativity of the generation model by directing its focus toward keywords. This not only enhances the quality of the generated output but also objectively limits the number of excessively generated triplets. Leverag- ing the semantic alignment with labels, TAGS fur- ther enhances the quality of the generated triplets. Consequently, compared to conventional genera- tion methods, our method can extract more correct triplets with fewer predicted triplets. This leads to higher precision, recall, and consequently, higher FI score. Furthermore, when compared to conven- tional sequence tagging methods, TAGS surpasses them due to the generation model's ability to uti- lize semantic information from the labels and its inherent creativity. Thus, TAGS outperforms most previous methods in terms of Fl score, precision, and recall.
Table 3: Ablation study. The results reported are the average Fl scores.
Model
16Res
15Res
141.ap
I4Res
Full Model
76.61
67.90
64.53
75.05
w/o Tagging traing
72.83
64.34
62.15
72.96
w/o Tag Attention
75.68
66.51
63.14
74.18
w/o Inference
75.49
66.66
64.19
74.34
w/o Alignment
75.37
65.82
63.20
73.44
4.3 Ablation
The results of the ablation experiments pre- sented in Table 3.
Effectiveness of the Sequence Tagging Task: The "w/o Tagging training" condition denotes the removal of the sequence tagging task, including multitask training, tag attention, and the specialized inference stage. means that the model only re- lies on the Semantic Alignment component. Com- pared to the "Full Model", the performance un- der this condition decreased all datasets: 16res

---

(-3.78%), 15res (-3.56%), 14Lap (-2.38%), and 14Res (-2.09%), providing evidence for the effec- tiveness of the sequence tagging task. To further investigate the role the sequence tagging task, we conducted Experiments 4.3.1.
Table 4: Fl results on the development datasct for dif- ferent thresholds.
threshold
16res
15res
14lap
14res
0.9
77.14
73.91
62.14
65.88
Effectiveness of Tag Attention: The "w/o Tag attention" condition refers to the absence of tag attention while still retaining the training of the sequence tagging task, special inference stage, and the Semantic Alignment component. When com- pared to the "full model," there was an average performance decrease of 1.11% % across all datasets, providing evidence for the effectiveness of Tag At- tention. In Appendix B.2, we further analyze the impact of different utilization methods for sequence tagging probabilitics Tag Attention. This anal- ysis enables gain a deeper understanding of how the utilization of sequence tagging probabili- ties influences the performance of Tag Attention. Effectiveness of Inference: The "w/o Infer- ence" condition refers to the absence of a spe- cial inference stage. In comparison to the "Full model," there was an average performance decrease of 0.85% across all datasets. This provides evi- dence for the effectiveness of the Inference stage. Experiment 4.3.2, we further investigate the ex- perimental results related to the threshold hyperpa- rameter in the inference stage.
0.99
77.62
74.27
62.61
66.00
0.999
77.73
74.35
62.78
66.08
0.9999
77.66
74.29
62.77
66.00
67.8
67.5
67.6
61.0
0y
66.8
0--11
4,
66.6
0y-2
30
15
20
030
0.75
1:00
1:25
150
0:
(a) generation ratio
(b) tagging ratio
Figure 2: Fi result with different loss ratios.
a3 achieves the best result. We select the loss ratios corresponding to the optimal performance as our final hyperparameter settings: æ1 = 10, 02 = 1, 03 = 1. This suggests that our method primarily focuses on the generative task, with the other two components serving as auxiliary factors.
4.3.2 Threshold Hyperparameter in Inference
Effectiveness of the Semantic Alignment: The "w/o Aligmment" condition refers to the removal of the Semantic Alignment component. Compared to the "Full Model", the performance under this condi- tion decreased in all datasets: 16res 11,224%) 15res (-2.08%), 14Lap (-1.33%), and 14Res (-1.61%) This demonstrates the effectiveness of the Semantic Alignment component improving overall perfor- mance. To further investigate the impact the loss function on the Semantic Alignment component, we conducted Experiment B.3.
We conducted experiments the development set to determine the most suitable probability thresh- old hyperparameter. We experimented with four different values for the threshold hyperparameter. The results are shown in Table 4. As the thresh- old increases, the performance initially improves and then decreases, achieving the best result at 0.999. This threshold value is very close to 1. In the generated results, the probability of each word is also very close even for some incorrect words. Therefore, when we require threshold filter out potentially erroneous triplets, this thresh- old should also be very close to 1. Hence, 0.999 is a reasonable choice.
4.3.1 Loss Hyperparameters
this section, we investigate the impact of loss hy- perparameters. First, we fix 02 and vary 01 and 03, as shown in Fig. 2(a). As oq gradually increases, the performance initially improves and then de- creases, achieving the best result at 10. Comparing the three curves in the graph, the curve correspond- ing 03 achieves the best result. Next, fix 01 10 and vary a2 and ag as shown in Fig. 2(b). As the 0g increases, the performance initially improves and then decreases, achieving the best re- sult at 1. Furthermore, the curve corresponding to
4.3.3 Correctness of Semantic Representation
To demonstrate that HY, is more accurate, during self-decoding, we replace each label sentence with the original input sentence with probability of r. This increases the influence of irrelevant words on semantic representation. We then train the TAGS model using the semantic representation obtained from this decoding process and the correspond-

---

Model
ldßes
141ap
15Res
14Ros
76.50
HAST ToWT-aa al., 2021b)
63.84
53.41
58.12
J.SEE MrS//hnn 2021b)
67.65
52.34
59.64
66.02
76.25
SpunMiiZhuoel 2020)
71.78
76.00
Shen et al. 2020)
23.67
66.18
66.75
73.30
GASiZhanng
75.75
LEGOX et 2022)
77.6
69.7
71.4
28.1
PHGU
TAGS (Our)
80.47
71.20
71.23
78.61
Table 5:
Main FI results of the AOPE task. The best
0.2
0.4
0.6
D.8
1.0
results are in bold, second best results are underlined.
Figure 3: Fi result with different
Model
14Lap
16Res
15Res
14Res
BERT+GRUILI et 2019b)
61.12
70.21
59.60
73.17
SPAN-BERT(Hu 2019)
61,25
73.68
MN-RERT (Li 2019h)
61.73
60.22
70.72
ing performance reflects the correctness of the se- mantic representation. We conducted experiments on the 16res dataset and the results presented in Fig. The results indicate that increases, performance decreases. This demonstrates that an increasing number of irrelevant words the input lead to a decrease in the correctness of the seman- tic representation, resulting in a gradual decline in performance.
RACL/Chen and Quan, 2020)
63:40
66.05
73.42
Dual-MRC(Maa et 2021)
65,94
65.08
75.95
GASiZhang al., 2021h)
67.37
71,87
65.75
75.77
EHGdL et al., 2023)
68.48
77.12
70.04
79.32
TAGS (Our)
71.37
78.11
70.76
79,80
Table 6: Main Fl results of the UABSA task. The best results are bold, second best results are underlined.
the accuracy sentiment analysis.
These results demonstrate the effectiveness and generalization of TAGS across different tasks.
4.4 Results on Other ABSA Tasks
The proposed model provides unified framework to effectively address the Aspect-Basce -asee Sentiment Analysis (ABSA) problem. To demonstrate the ef- fectiveness of TAGS and its generalizability across different tasks, we conducted experiments on two ABSA tasks: AOPE and UABSA. We compared TAGS with the models in Appendix B.4.
Related Work
ASTE employed sequence tagging methods, when was first introduced by (Peng et al., 2020). Subse- quent research efforts (Xu et al., 2020b; Wu et al., 2020; Chen et al., 2022a; Liang al., 2022; Gou et al., 2023) have been focused enhancing the sequence tagging schemes and model components to facilitate the integration and mutual interpre- tation of the triple elements. However, the se- quence tagging technique in ASTE fails to capture the semantic information conveyed by the labels, which can lead to semantic mismatches in the pre- dicted results(Zhang et al., 2021b). Generation methods were initially proposed by Zhang et al. (2021c). The interatioo-based approach in ASTE has achieved good performance by reducing poten- tial error propagation present in pipeline methods and effectively utilizing the rich semantic infor- mation provided by labels(Paolini et al., 2021; Yu et al., 2023). They employed various targets for generation, such as sentiment element sequences (Zhang et al., 2021c,c; Hu et al., 2022b), natural language (Liu et al., 2021; Zhang et al., 2021a), and structured extraction patterns (Lu al., 2022). Re- cently proposed models, LEGO-ABSA (Gao al., 2022), UnifiedABSA (Wang et al., 2022) and Mvp (Gou et al., 2023), have focused on leveraging task prompts or guided design for multi-tan taikk processing.
AOPE focuses on extracting (aspect, opinion) pairs, similar ASTE, but without sentiment anal- ysis. This task requires accurate identification of keywords in the sequence tagging task, as well as the assistance of Tag Attention and Semantic Align- ment components. The Fl results for the AOPE task are presented in Table 5. TAGS outperforms the previous model on all four datasets: 2.28% for 16Res, 1 1.50% for 14Lap, 0.83% for 15Res, and 0.51% for 14Res. The improvement in results demonstrates the effectiveness of the aforemen- tioned components.
UABSA focuses on extracting (aspect, senti- ment) pairs, similar to ASTE, but without extract- ing opinions. This task presents challenges in accu- rately classifying sentiments in sequence tagging and aligning sentiments in Semantic Alignment. The Fl results for the UABSA tasks are presented in Table 6. TAGS has achieved an average improve- ment of 1.27% compared the previous model. This improvement demonstrates the effectiveness of the aforementioned components in enhancing

---

6 Conclusion
8 Ethics Statement
this work, we introduce a generation model called TAGS, which enhances the supervision of both the encoder and decoder through multiple- perspective tagging assistance and label semantic representations. Specifically, TAGS utilizes se- quence tagging to enhance the generation model in multiple aspects: Multitask Learning, Guided Generation, and Result Optimization. Additionally, TAGS employs label self-decoding process ob- tain semantic representations of labels and aligns the decoder's hidden states with these representa- tions, thereby providing enhanced semantic super- vision for the decoder's 's hidden states. These two components enhance the supervision of the encoder and decoder's hidden states, resulting in improved generation quality. Extensive experiments demon- strate that our method significantly advances the state-of-the-ee on benchmark datasets.
In all our experiments, used existing datasets that have been widely used in previous scientific publications. When analyzing the experimental results, we strive to maintain fairness and honesty, ensuring that our work does not cause harm to anyone.
As for broader implications, this work may con- tribute to further research in sentiment analysis and the use of generation methods to simplify and automate the extraction of user opinions in real- world applications. However, it important note that this work involves fine-tuning large-scale pre-trained language models to generate sentiment triplets. Due to the nature of the Internet-base large-scale pre-training corpora, the predicted sen- timent polarities may be influenced by unintended biases related to gender, race, and intersectional identities (Tan and Celis, 2019). LPMLS often in- herit biases present in their training data, potentially leading to biased sentiment analysis results, partic- ularly when assessing text from underrepresented or marginalized groups, thereby perpetuating and amplifying societal prejudices. Another limitation is the opacity of these models. Their complex archi- tectures make it challenging to fully understand the reasoning behind their predictions, raising concerns about transparency and accountability. This lack of interpretability may hinder the identification and mitigation of harmful biases and ethical violations in sentiment analysis applications. is crucial for the natural language processing community to con- sider these biases more extensively. Fortunately, these issues are actively being addressed within the research community, including efforts to standard- ize datasets and methodologies.
7 Limitations
Despite achieving statc-of-ther suc-thhe-rt art performance, our proposed methods still have some limitations that point to potential future directions.
1. Compared to conventional generation meth- ods, our approach requires an additional gen- eration step to obtain more accurate hidden states, namely semantic labels. As a result, there is an increase in training overhead.
2. Although we apply a simple yet effective ag- gregation strategy combine the results of the sequence tagging task and generation task, more advanced strategies can be explored to further enhance performance.
Acknowledgements
3.
We have indeed observed that the improve- ment of our model varies different datasets, which may be due the differences in the characteristics of these datasets.
We would like to thank the anonymous review- ers for their insightful comments. This work is partially supported by National Natural Science Foundation of China (Grants No. 62176271), and Science and Technology Program of Guangzhou (Grant No. 202201011681).
4. Our work utilizes a relatively simple sequence tagging approach, specifically characterized by the absence of explicit pairing between tracted aspects and opinions. There is room for designing more robust and sophisti- cated sequence tagging scheme that can also seamlessly integrate with generation models, thereby enhancing performance.
References
Hongjie Cai, Rui Xia, and Jianfei Yu. 2021. Aspect- sinegorr--piniion-eeiieenn quadruple extraction with implicit aspects opinions. Proceedings of the 59th Annual Meeting of the Association for

---

Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 340-350, Online. Association for Computational Linguisties.
Tianhao Gao, Jun Fang, Hanyu Liu, Zhiyuan Liu, Chao Liu, Pengzhang Liu, Yongjun Bao, and Weipeng Yan. 2022. LEGO-ABSA: A prompt-hased task assem- blable unified generative framework for multi-task aspect-based sentiment analysis. In Proceedings of the 29th International Conference on Computational Linguistics, pages 7002-7012, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Hao Chen, Zepeng Zhai, Fangxiang Feng, Ruifan Li. and Xiaojie Wang. 2022a. Enhamced multi- chati-c graph convolutional network for aspect sentiment triplet extraction. Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27. 2022, pages 2974-2985. Associ- ation for Computational Linguistics.
Zhibin Gou, Qingyan Guo, and Yujuu Yang. 2023. Mvp: Multi-view prompting improves aspect sentiment tu- ple prediction. CoRR, abs/2305.1200 126277.
Shaowei Chen, Jie Liu, Yu Wang, Wenzheng Zhang, and Ziming Chi. 2020. Synchronous double-channel recurrent network for aspect-opinion pair extraction. In Proceedings of the 58th Annual Meeting of the As- sociation for Computational Linguistics, pages 6515- 6524, Online. Association for Computational Lin- guistics.
Ruidan He, Wee Sun Lee, Hwee Tou Ng. and Daniel Dahlmeier. 2019. An interactive multi-task learning network for end-to-end to-d aspect-based based sentiment anal- ysis. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 504-515, Florence, Italy. Association for Computa- tional Linguisties.
Shaowei Chen, Yu Wang, Jie Liu, and Yuelin Wang. 2021. Bidirectional machine reading comprehension for aspect sentiment triplet extraction. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Thiie Conference on Innovative Ap- plications of Artificial Intelligence, IAAI 2021, The Fleventh Symposium on Educational Advances in Ar- rificial Intelligence, EAAI 2021, Virtual Event, Febru- ary 2-9, 2021, pages 12666-126660 AAAI Press.
Mengting Hu, Yike Wu, Hang Gao, Yinhao Bai, and Shi- wan Zhao. 2022a. Improving aspect sentiment quad prediction via template-order order data augmentation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 1889-79001 Association for Com- putational Linguistics.
Mengting Hu, Yike Wu, Hang Gao, Yinhao Bai, and Shiwan Zhao. 2022b. Improving aspect sentiment quad prediction via template-order data augmenta- tion. EMNLP.
Yugi Chen, Keming Chen, Xian Sun, and Zequn Zhang. 2022b. A span-level bidircctional network for as- pect sentiment triplet extraction. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirutes, December 7-11, 2022. pages 4300-4309. Association for Computational Linguis- tics.
Minghao Hu, Yuxing Peng. Zhen Huang, Dongsheng Li, and Yiwei Lv. 2019. Open-domain targeted senti- ment analysis via span-based extraction and classifi- cation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 537-546, Florence, Italy, Association for Computa- tional Linguistics.
Zhuang Chen and Tieyun Qian. 2020. Relation-aware collaborative learning for unified aspect-based sen- timent analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics. pages 6685-3694, Online. Association for Computational Linguistics.
Samson Yu Bai Jian, Tapas Nayak. Navonil Majumder, and Soujanya Poria. 2021. Aspect sentiment triplet extraction using reinforcement learning. In CIKM '21: The 30th ACM International Conference Infor- mation and Knowledge Management, Virtual Event, Queensland, Austrulia, November 1 - 5, 2021, pages 9603-3607. ACM.
Zhifang Fan, Zhen Wu, Xin-Yu Dai, Shujian Huang, and Jiajun Chen. 2019. Target-oriented opinion words extraction with target-fused neural sequence labeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume (Long and Short Pupers), ). pages 2509-25 18, Minncapolis, Minnesota. Association for Computa- tional Linguistics.
Xin Li, Lidong Bing, Piji Li, and Wai Lam. 2019a. A unified model for opinion target extraction and target sentiment prediction. In The Thirty-Third AAAI Con- ference Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial In- telligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27- February 1, 2019, pages 6714-6721. AAAI Press.
Hao Fei, Fei Li, Chenliang Li, Shengqiong Wu, Jingye Li. and Donghong Ji. 2022. Inheriting the wisdom of predecessors: A multiplex cascade framework for unified aspect-based sentiment analysis. In Proceed- ings of the Thirty- First International Joint Confer- ence on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022, pages 4121-4128. ojcai.org.
Xin Li, Lidong Bing. Wenxuan Zhang, and Wai Lam. 2019b. Exploiting BERT for end-to-end aspect-based

---

sentiment analysis. In Proceedings of the 5th Work- shop on Noisy essr-generated Text (W-NUT 2019), pages 34-41, Hong Kong, China. Association for Computational Linguistics.
pointer networks for aspect sentiment triplet extrac- tion. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Domini- can Republic, 7-11 November, 2021, pages 9279- 9291. Association for Computational Linguisties.
Shuo Liang, Wei Wei, Xian-Ling Mao, Yuanyuan Fu, Rui Fang, and Dangyang Chen. 2022. STAGE: span tagging and greedy inference scheme for aspect sen- timent triplet extraction. CoRR, abs/221 150003.
Giovanni Paolini, Ben Athiwaratkum, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, Cícero Nogueira dos Santos, Bing Xiang, and Ste- fano Soatto. 2021. Structured prediction as transla- tion between augmented natural languages. In 9th International Conference on Learning Representa- tions, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Shuo Liang, Wei Wei, Xian-Ling Mao, Yuanyuan Fu, Rui Fang, and Dangyang Chen. 2023. STAGE: span tagging and greedy inference scheme for aspect senti- ment triplet extraction. In Thirty-Seventh AAAI Con- ference Artificial Intelligence, AAAI 2023, Thirty- Fifth Conference on Innovative Applications of Artifi- cial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, pages 13174-13182. AAAI Press.
Haiyun Peng, Lu Xu, Lidong Bing, Fei Huang, Wei Lu, and Luo Si. 2020. Knowing what, how and why: A near complete solution for aspect-based sen- timent analysis. In The Thirty-Fourth AAAI Con- ference Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial In- telligence, EAAI 2020, New York, NY. USA, February 7-12, 2020. pages 8600-8600 8607. AAAI Press.
Jian Liu, Zhiyang Teng, Leyang Cui, Hammeng Liu, and Yue Zhang. 2021. Solving aspect category sentiment analysis as a text generation task. In Proceedings of the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 4406 4406-4416, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Joseph Peper and Lu Wang. 2022. Generative aspect- based sentiment analysis with contrastive learning and expressive structure. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 6089-6095. Association for Computa- tional Linguistics.
Yaojie Lu, Qing Liu, Dai Dai. Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu, 2022. Uni- fied structure generation for universal information extraction. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5755-5772, Dublin, Ireland. Association for Computational Linguistics.
Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Moham- mad AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clereq, Véronique Hoste, Maarnanna Apidianaki, Xavier Tannier, Na- talia Loukachevitch, Evgeniy Kotelnikov, Nuria Bel, Salud María Jiménez-Zafra, and Gülşen Eryigit. 2016. SemEval-2016 task 5: Aspect based sentiment analysis. Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-201e) pages 19-30, San Diego, California. Association for Computational Linguistics.
Haoran Lx, Junyi Liu, Henan Wang, Yaoming Wang, Jix- iang Luo, and Yaxiao Liu. 2023. Efficient hybrid gen- eration framework for aspect-based sentiment anal- ysis. In Proceedings of the 17th Conference of the European Chapter of the Association for Computa- tional Linguistics, EACL 2023, Dubrovnik, Croatia, May 2-6, 2023, pages 1007-1018. Association for Computational Linguistics.
Yue Mao, Yi Shen, Jingchao Yang, Xiaoying Zhu, and Longjun Cai. 2022. Seq2path: Generating sentiment tuples as paths of a tree. In Findings of the Associa- tion for Computational Linguistics: ACL 2022, pages 2215-2225.
Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015 5. SemFval--2015 task 12: Aspect based sentiment anal- ysis. In Proceedings of the 9th International Work- shop on Semantic Evaluation (SemEval 2015), pages 486-495, Denver, Colorado. Association for Compu- tational Linguistics.
Yue Mao, Yi Shen, Chao Yu, and Longjun Cai. 2021. joint training dual-mr mre framework for aspect based sentiment analysis. In Thirty-Fifthh AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence EAAI 2021, Virtual Event, February 2-9, 2021, pages 1554--1355) AAAI Press.
Maria Pontiki, Dimitris Galamis, John Pavlopoulos, Har- ris Papageorgiou, lon Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 task Aspect based sentiment amalysis. In Proceedings of the 8th International Workshop Semantic Evaluation (Se- mEval 2014), pages 27-35, Dublin, Ireland. Associa- tion for Computational Linguistics.
Rajdeep Mukherjee, Tapas Nayak, Yash Butala, Sourangshu Bhattacharya, and Pawan Goyal. 2021. PASTE: A tagging-free decoding framework using

---

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res.,
Li Yuan, Jin Wang, Liang- Liang-Chih Yu, and Xuejie Zhang. 2023. Encoding syntactic information into trans- formers for aspect-based sentiment triplet extraction. IEEE Transactions on Affective Computing, pages 1-15.
Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Ling- peng Kong, and Nigel Collier. 2022. A contrastive framework for neural text generation. In NeurIPS.
Chen Zhang. Qiuchi Li, Dawei Song, and Benyou Wang. 2020. A multi-task learning framework for opinion triplet extraction. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of Findings of ACL, pages 519-828. Association for Computational Linguistics.
Yi Chern Tan and L. Elisa Celis. 2019, Assessing so- cial and intersectional biases in contextualized word representations. In Advances in Neural Information Processing Systems 32: Annual Conference Neu- ral Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 13209-13222 13220.
Mi Zhang and Tieyun Qian. 2020. Convolution over hi- erarchical syntactic and lexical graphs for aspect level sentiment analysis. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 3540-3549, Online. As- sociation for Computational Linguistics.
Zengzhi Wang, Rui Xia, and Jianfei Yu. 2022. Unified- absa: A unified ABSA framework based on multi- task instruction tuning. CoRR, abs/2211 10996....
Wenxuan Zhang, Yang Deng, Xin Li, Yifci Yuan, Li- dong Bing, and Wai Lam. 2021a. Aspect sentiment quad prediction as paraphrase generation. ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7- 11 November, 2021, pages 9209-92999911 9219, Association for Computational Linguistics.
Zhen Wu, Chengean Ying, Fei Zhao, Zhifang Fan, Xinyu Dai, and Rui Xia. 2020. Grid tagging scheme for aspect-orientee fine-grained opinion extraction. Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 2576-2585, Online. Association for Computational Linguistics.
Pro-
Lu Xu, Yew Ken Chia, and Lidong Bing. 2021. Learn- ing span-level interactions for aspect sentiment triplet extraction. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Natu- val Language Processing (Volume I: Long Papers), pages 4755-4766. Online. Association for Computa- tional Linguisties.
Wenxuan Zhang, Xin Li. Yang Deng, Lidong Bing, and Wai Lam. 202 Towards generative aspect-based sentiment analysis. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing, ACI/IICNIPP 2021, ( (Volume 2: Short Papers), Virtual Event, Au- gust 1-6, 2021, pages 504-510. Association for Com- putational Linguistics.
Lu Xu, Hao Li, Wei Lu, and Lidong Bing. 2020a. Position-aware tagging for aspect sentiment triplet extraction. In Proceedings the 2020 Conference on Empirical Methods in Natural Language Processing. EMNLP 2020, Online, November 16-20, 2020, pages 2339-2349. Association for Computational Linguis- tics.
Wenxuan Zhang, Xin Li, Yang Deng, Lidong Bing, and Wai Lam. 2021c. Towards generative aspect-based sentiment analysis. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 2: Short Papers), pages 504-510, Online. Association for Computational Linguistics.
Lu Xu, Hao Li, Wei Lu, and Lidong Bing. 2020b. Position-awas aware tagging for aspect sentiment triplet extraction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2339-2349 Online. Association for Computational Linguistics.
Yice Zhang, Yifan Yang. Yihui Li, Bin Liang, Shi- wei Chen, Yixue Dang, Min Yang. and Ruifeng Xu. 2022a. Boundary- driven table- filling for aspect senti- ment triplet extraction. In Proceedings of the 2022 Conference Empirical Methods in Natural Lan- guage Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 6485- 6498. Association for Computational I. Linguisties.
Hang Yan, Junqi Dai, Tuo Ji, Xipeng Qiu, and Zheng Zhang. 1021. A unified generative framework for bssedt-based sentiment analysis. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1ith International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 242992 Online. Association for Computational Linguistics.
7hihao Zhang, Yuan Zuo, and Junjie Wu. 2022b. Aspect sentiment triplet extraction: A seq2seq approach with span copy enhanced dual decoder. IEEE ACM Trans. Audio Speech Lang. Process., 30:2729-2742.
Chengze Yu, Taiqiang Wa, Jiayi Li, Xingyu Bai, and Yujiu Yang. 2023. Syngen: A syntactic plug-and- play module for generative aspect-based sentiment analysis. CoRR, abs/2302.13032.
He Zhao, Longtao Huang, Rong Zhang, Quan Lu, and Hui Xue. 2020. SpanMIt: A span-based multi-task

---

Table Descriptions of the tagging scheme. focuses the aspect attribution of each word. The sequence tagging task is to classify each word to one of these tags.
A.3
Other Attention
0hi crp(ac)
(13)
1== expp(a)))
tag
Mcaning
caplps a1k)
N
not belong to aspect term or opinion term
01 Σ j=1 caplpj atj)
(14)
A-POS
part of aspect term with positive sentiment
A-NEG
a part of aspect term with negative sentiment
B Additional Experiment
A-NEU
part of aspect term with neural sentiment
B.1 case study
O-POS
a part of opinion term with positive sentiment
In this case study section, compare our model with the Paraphrase model (Zhang et al., 2021a) to illustrate how our two components benefit the results. For the first example, the Paraphrase model fails to extract the triplet (barmenu, Disappointingly, N EG) because it is relatively hard and implicit triplet. Addition- ally, thanks to the semantic alignment training of the decoder hidden state, our model can generate higher-quality results. Therefore, our model can extract this triplet successfully. In the second ex- ample, the Paraphrase model incorrectly extracts the triplet (staf supportive, N FG). However, during the inference stage, our model optimizes the generation results based on the sequence tagging output, resulting in the discarding of this incorrect triplet.
O-NEG
part opinion with negative sentiment
O-NEU
a part of opinion term with neural sentiment
learning framework for pair-wise aspect and opinion terms extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 3239-3248 3248, Online. Association for Computational Linguistics.
Additional details about the methodology
A.1 Tagging Scheme
Specific details of the sequence tagging scheme and their explanations are presented in Table 7.
A.2 Inference
B.2 Arithmetic Operations in Tag Attention In the context of Tag Attention, we have explored several approaches incorporating tagging proba- bility into the cross-attention mechanism:
Here's an overview of the inference process:
1. Obtain the probabilities of cach word in the generated sentence.
Multiplication before softmax: Multiply the attention scores by the probability weights and then apply softmax.
2. Conduct experiments on the development dataset to find a suitable probability thresh- old.
2. Multiplication is performed before softmax, but without adding 1 to pi. The attention for- mula is given by Equation (14).
3. For each triplet in the generated result, check if all the words in the triplet have probabilities greater than the threshold. If so, retain the triplet. Otherwise, proceed to the next step.
3. Softmax after multipicaation: Apply sofumax to the attention scores and then multiply them by the probability weights.
4. Compare the generated triplet with the triplet identified by the sequence tagging task. the generated aspect is a subset of the sequence tagging aspect set, vice versa, and the gen- erated opinion is subset of the sequence tagging opinion set, or vice versa, retain the triplet. Otherwise, discard the triplet.
4. Addition: We directly add the probability in- formation the attention scores.
Through the evaluation of these various opera- tions, objective gain insights into their impact on the Tag Attention mechanism and their effectiveness in incorporating probability informa- tion. We conducted this experiment on a model without the "Inference" process because including
The specific algorithm pseudocode is presented in algorithm I.

---

Table 8: Case study. The ground truth represents the correct triplets. The aspect and opinion words of the same triplet are highlighted in the same color. The two examples in the table demonstrate how model can avoid the errors made by the Paraphrase model.
Example
Groand Truth
Paraphrase (Zhong al, 2021n)
Ours
Deoppiintnng, their wonderful Saketini los taken bar
Sukkriiii,sndeerti POSS)
(Salueini wonderful.POS
(Sakettini wwnderful POS;
hewp:n.ioopppnnnggg..o. NEC)
walter friendly and auppertine
(walter Iriendly, PUS)
(walter Friendly POS) (saff supfssppetsss
(walter friendllyyyyyyC PNs)
Table 9: Result of different arithmetic operations
Table 10: Result of different loss function
16res
15res
14lap
14res
16res
15res
14lap
14res
Multiplication before softmax
76.61
67.90
64.53
75.05
MarginRankLoss.
75.55
67.12
64.19
73.91
Multiplication before softmax
75.81
66.71
63.23
74.09
without adding to Softmax after multiplication Addition
BCELoss
76.61
67.90
64.53
75.05
73.92
65.05
61.78
72.38
73.06
64.58
61.43
72.12
no need to excessively move away from the nega- tive hidden states associated with incorrect words. Since the hidden states are generated from the la- bel sentences, even if some negative hidden states are incorrect, they remain relatively close to the correct hidden states. Moving too far away from negative hidden states may lead to an increase in the distance from the correct hidden state.
the "Inference" " process could potentially narrow down the performance gaps observed in these re- sults. The results in Table 9 demonstrate that the first approach performs better. This is because it provides valuable information to the generation model while minimizing any disruptive effects on the original generation process. can be regarded as gentle process. The results of the second ap- proach are worse compared to the first approach. One possible reason for this is that in the first ap- proach, by adding 1 to the p. the attention is not solely determined by the sequence tagging results. This helps mitigate the potential impact of tagging errors the overall generation quality. Further- more, we found that the performance of the last two arithmetic operations is worse.
B.4 ABSA subtask Detail
The subtasks are described as follows:
Aspect Opinion Pair Extraction (AOPE) aims to extract aspect terms and their corresponding opinion terms as pairs (Zhang and Qian, 2020; Chen et al., 2020).
2. Unified ABSA (UABSA) is the task of ex- tracting aspect terms and predicting their sen- timent polarities at the same time (Li et al., 2019a; Chen and Qian, 2020). We also formu- late it as an (aspect, sentiment polarity) pair extraction problem
B.3 Loss Function for Semantic Alignment
We discuss loss function for semantic alignment in our approach. Specifically, we compare two different approaches:
For these tasks, we adopt the dataset used in (Zhang et al., 2021b).
1. Confining the similarity scores to the range of and utilizing the Binary Cross Entropy (BCE) loss function.
For AOPE task, we compare our model with the following models: a multi-task learning model SpanMIt (Zhao et al., 2020), synchronous dou- ble channel extraction model SDRN (Chen et al., 2020), HAST+TOWE and JERE-MHS model com- pared in (Zhang et al., 2021b), GAS (Zhang et al., 2021b), LEGO et al., 2022) and EHG(Lv et al., 2023).
2. Preserving the cosine similarity scores in the range of -1 1 and employing the margin ranking loss function to constrain the similar- ity.
The results in Table 10 indicate that in our method, the BCE loss function outperforms the margin rank loss function. From this, we can con- clude that not necessary to push the similar- ity of incorrect hidden states to -1, i.e, there is
For the UABSA task, we compare our model with the following models: a BERT base model BERT+GRU (Li al., 2019b), a span-base extrac- tion model SPAN-BER BERT (Hu et al., 2019), an in- teractive multi-tast learning network LMN-

---

(He et al., 2019), Relation-Aware Collaborative Learning (RACL) model RACL (Chen and Qian, 2020), a machine reading comprehension models Dual-MRC (Mao et al., 2021), GAS (Zhang et al., 2021b) and E et al., 2023).
B.5 Analysis on Potential Practical Applications
Time Complexity:
The time complexity of the TAGS model is quadratic relative to the input data. The primary source of complexity in this quadratic time complexity is the attention operations within the transformer. It's important to note that the ad- ditional modules introduced our model, such as the sequence tagging classification layer and the label-driven semantic alignment module, have linear time complexity relative to the input data. Consequently, the time complexity introduced by our additional modules remains exceedingly mod- est compared that of the transformer. As such, the primary temporal overhead in our model stems from the transformer's attention operations. Conse- quently, the complexity of the TAGS model closely aligns with the time complexity baseline mod- els that rely on transformers. Moreover, exist- ing lightweight and acceleration-oriente designs based on the transformer can be readily assimilated into model. Hence, although model does introduce some additional time overhead, does not impose significant obstacle to the training process.
Space Complexity: Apart from the core model architecture and input data, the additional space utilization of the TAGS model primarily consists of a linear layer for sequence tagging classification and the semantic representation of label sentences. The additional space occupation amounts 5.2 M, which is notably minor when compared to the parameter size the T5 model, standing at 222 M. Additionally, the tag attention module does not introduce any additional parameters.
Based on the aforementioned explanation, it's evident that the TAGS model demonstrates com- mendable scalability. As dataset volumes increase, the incremental rise in both time and space over- heads within our model remains consistent.

---

Input:
Generated triplets Ti- {a O₁, 8: si)k
1771
k=1"
Ti
Word probabilities G-(gk= (g 9k2, Dk3) )s
1=
(Sapprtt
Tagging Aspect Set Suspect = {a() k=1
Tagging Opinion Set Sopinion = {ap)
Sepinion k=1
Threshold threshold.
Output: Result triplets
Function Veri Verif (egeeeeeeeemeeent element, sequenceTag): foreach tag_element in sequenceTag do
if gen_enn element is part of tag__alegeeeae eleeeeeent OR tag elemeet is a part of gen eimeeeemnt then return true
end end
return false begin
Result
foreach (triplet, wordProb) in (Ti G)) do (wordProb 2 threshold) then Result< Result U (triplet) end else
aspect arrppeetspect
opinion triplet.ionnnlllll......iooo
Overified< herpoddiiiiirilliiiiiooo...n if Averified AND Overified then Result-- Result (triplet)
end
end
end
end
return Result
Algorithm 1: Inference


Reviews:
[{"review_id": "f86MOT3Bhm", "reviewer": "Reviewer_Gsym", "summary": "", "questions": "Why Chen et al., 2022b is not included for comparison?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our sincere gratitude to you for your valuable feedback and insights into our work. \n\n**For question #1 \"Why Chen et al., 2022b is not included for comparison?\":**\n\nWe apologize for not comparing with the work by Chen et al., 2022b in our paper. It was an oversight on our part. The results of the work by Chen et al., 2022b are listed in Table 1 and discussed in the following paragraphs. We will include the comparison with Chen et al., 2022b in the revised version of our paper.\n\n**For the \"Missing References\"**: \n\nWe appreciate your suggestion to include more ASTE tagging methods for comparison. We have listed the results of these methods in Table 1 and discussed them in the following paragraphs. In our revised paper, we will incorporate a comparative analysis of our TAGS model and these additional methods, along with proper citations. This will ensure a more comprehensive evaluation of the TAGS model's performance in ASTE tagging methods.\n\nTable 1 provides a concise comparison of the F1 performance results of our TAGS model and the works the reviewer referred to. The F1 performance is extracted from the respective publications of Chen et al., 2022b, and the three papers mentioned by the reviewers for comparison.\n\nTable 1. Comparison with Chen et al., 2022b and other ASTE tagging methods \n|       Dataset      | 16res | 15res | 14lap | 14res |\n|:------------------:|:-----:|:-----:|:-----:|:-----:|\n|    Chen et al., 2022b | 72.08 | 64.82 | 62.65 | 74.34 |\n|        the first paper | 72.76 | 63.13 | 60.11 | 74.84 |\n|  the second paper    | 68.64 | 61.54 | 60.95 | 71.62 |\n|   the third paper    | 67.13 | 60.48 | 54.44 | 68.22 |\n|     TAGS(ours)     | **76.61** | **67.90** | **64.53** | **75.05** |\n\nThe results table illustrates that the TAGS model outperforms the results achieved by Chen et al. (2022b) to a significant degree (e.g., over 4% improvement in the case of 16res). This is largely attributed to the TAGS model's superior ability, in comparison to Chen's model, to comprehensively consider contextual information when pairing aspect and opinion elements. \n\nFurthermore, the TAGS model outperforms the three sequence tagging models. For instance, compared to the first paper (the best one) of the three sequence tagging methods paper, about 4% improvements are achieved in the former three datasets. Our model demonstrates superiority over these sequence tagging models, attributed to the advantage of our generation task, which is assisted by tagging and aligned by the label-driven semantics. \n\nFinally, it is important to note that the supplementary comparisons we have provided will not impact the experimental conclusions in our manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper presents a new ASTE generation model and outperforms the SOTA methods.", "reasons_to_reject": "None", "questions_to_authors": "Why Chen et al., 2022b is not included for comparison?", "paper_topic_and_main_contributions": "This paper presents a tagging-assisted generation model with encoder and decoder supervision (TAGS) for the ASTE task. The contribution is a new ASTE generation model with the SOTA results.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "More ASTE tagging methods should be included for comparison.\n1. https://www.ijcai.org/proceedings/2022/572\n2. https://ieeexplore.ieee.org/document/9857593\n3. https://ieeexplore.ieee.org/document/10175600"}}, {"review_id": "9cXyR6z2Yf", "reviewer": "Reviewer_3TTV", "summary": "", "questions": " 1.What potential biases may affect the predicted sentiment polarities in this work? \n 2.Does the article mention any limitations or challenges of the sequence tagging approach used in this work? \n 3.Does the article discuss any potential ethical concerns related to the use of large-scale pre-trained language models in sentiment analysis? ", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our sincere gratitude to you for your valuable feedback and insights into our work.\n\n**For the reason to reject#1 \"The paper does not provide a detailed discussion of the potential practical applications of the proposed TAGS model, such as its scalability or its compatibility with different types of data.\":**\n\nWe appreciate the reviewer's comment regarding the need for a detailed discussion of the potential practical applications, such as scalability, and compatibility of the proposed TAGS model. **In fact, we have already voiced concerns about the additional time overhead associated with the process of obtaining semantic representation of labels in our \"Limitations\" section: \"Compared to conventional generation methods, our approach requires an additional generation step to obtain more accurate hidden states, namely semantic labels. As a result, there is an increase in training overhead.\" However, it is worth emphasizing that these supplementary time costs do not significantly impede the model's training process.**\n\nIn our revised manuscript, we will provide the following comprehensive discussions of these critical aspects.\n\n**Scalability Analysis:**\n\n**Time Complexity:** The time complexity of the TAGS model is quadratic relative to the input data. The primary source of complexity in this quadratic time complexity is the attention operations within the transformer. It's important to note that the additional modules introduced in our model, such as the sequence tagging classification layer and the label-driven semantic alignment module, have a linear time complexity relative to the input data. Consequently, the time complexity introduced by our additional modules remains exceedingly modest compared to that of the transformer. As such, the primary temporal overhead in our model stems from the transformer's attention operations. Consequently, the complexity of the TAGS model closely aligns with the time complexity of baseline models that rely on transformers. Moreover, existing lightweight and acceleration-oriented designs based on the transformer can be readily assimilated into our model. Hence, although our model does introduce some additional time overhead, it does not impose a significant obstacle to the training process.\n\n**Space Complexity:** Apart from the core model architecture and input data, the additional space utilization of the TAGS model primarily consists of a linear layer for sequence tagging classification and the semantic representation of label sentences. The additional space occupation amounts to 5.2 M, which is notably minor when compared to the parameter size of the T5 model, standing at 222 M. Additionally, the tag attention module does not introduce any additional parameters. \n\nBased on the aforementioned explanation, it's evident that the TAGS model demonstrates commendable scalability. As dataset volumes increase, the incremental rise in both time and space overheads within our model remains consistent and manageable.\n\nDuring practical training processes on datasets of varying scales, our model consistently maintains its performance without any noticeable degradation or efficiency concerns. In terms of computational resources, our TAGS model showcases a modest utilization of GPU memory. For instance, when training on the largest dataset among the four datasets used in our paper, utilizing a 24GB Nvidia 3090 GPU, we can set a batch size of 32. This emphasizes the minimal GPU memory demand of our model. Notably, for the largest dataset among the four, our TAGS model completes training an epoch in less than two minutes. This demonstrates the model's efficiency in terms of training time. Consequently, our TAGS model is well-equipped to support the training of large-scale datasets.\n\n**Compatibility with Different Types of Data:**\n\nThe datasets employed in our study, which are also widely recognized datasets, encompass three restaurant review datasets and one laptop review dataset. Sentiment analysis can be applied to a broader range of textual forms, including news commentary and various other domains. Our model exhibits adaptability to a wide spectrum of domains, given that the design of our TAGS model is domain-agnostic and does not rely on domain-specific prior knowledge.\n\nHowever, existing datasets within the field are predominantly centered around product reviews and lack diversity in terms of text forms, such as news commentary or other textual styles. As a result, conducting direct evaluations on these unexplored data types becomes unfeasible. Nevertheless, we undertook supplementary experiments on datasets from a recent ACL 2023 paper (released in May) to investigate TAGS' compatibility with a broader range of data types. (In adherence to the guidelines, I refrain from providing a direct link during rebuttal.)\n\nThis new dataset encompasses social media text and product reviews, distinguishing itself from the datasets used in our paper in several aspects:\n\n**Longer Texts:** The text in this new dataset is notably longer, with an average length approximately 3.6 times that of the commonly used datasets. **Longer sequence lengths in the dataset also facilitate the assessment of our model's scalability when presented with lengthier input data.**\n\n**Diverse Expressions:** The vocabulary within this new dataset is three times larger than that of the commonly used datasets, implying greater diversity and complexity. Moreover, it features more intricate grammar.\n\n**Varied Domains:** The dataset encompasses a multitude of domains, including electronics, beauty, fashion, and home, highlighting its compatibility with a wider array of domains.\n\nWe have conducted performance evaluations of the TAGS model on this new dataset and compared it with several models, with the F1 results obtained from the dataset's associated paper.\n\nTable 1. Evaluation of a new dataset from a recent ACL 2023 paper.\n| Dataset    | Electronics | Beauty | Fashion | Home  |\n|------------|-------------|--------|---------|-------|\n| BMRC       | 41.95       | 38.57  | 44.87   | 41.18 |\n| BART-ABSA  | 43.38       | 41.13  | 43.89   | 40.56 |\n| GAS        | 47.10       | 44.32  | 47.80   | 47.22 |\n| TAGS(ours) | **49.97**       | **47.12**  | **50.66**   | **50.24** |\n\nOur results significantly outperform the baseline comparisons, demonstrating improvements of 2.87% (Electronics), 2.8% (Beauty), 2.86% (Fashion), and 3.02% (Home) compared to the GAS model. This underscores the effectiveness of our contributions across multiple domains.\n\nIn summary, we will extensively expound upon the potential practical applications, such as scalability, and compatibility of the TAGS model in our revised manuscript. We firmly believe that these insights will fortify the comprehensiveness of our work.\n\n**For the reason to reject#2 \"The paper does not compare the proposed TAGS model with a wider range of state-of-the-art methods for ASTE, which could provide a more comprehensive evaluation of its effectiveness.\":**\n\n**We have indeed conducted comprehensive comparisons with various state-of-the-art (SOTA) methods, encompassing generation models, sequence labeling models, span-level models, and those based on reading comprehension. Our objective has been to provide an extensive evaluation that encompasses a diverse range of competing methodologies.**\n\nIn the revised version of our paper, we are committed to significantly expanding the comparative assessment section. We have diligently sought a broader array of comparative methods within the ASTE domain to ensure a more thorough evaluation of the TAGS model. For instance, we will introduce two additional comparative methods in our study, namely \"STAGE: Span Tagging and Greedy Inference Scheme for Aspect Sentiment Triplet Extraction\" and \"Boundary-Driven Table-Filling for Aspect Sentiment Triplet Extraction\". The following table presents a comparison of our model's F1 performance with that of the two papers. These results have been excerpted from their respective publications. \n\nTable 2. Comparison with two additional SOTA methods.\n|   Dataset  | 16res | 15res | 14lap | 14res |\n|:----------:|:-----:|:-----:|:-----:|:-----:|\n|    STAGE | 72.75 | 64.79 | 61.88 | 73.61 |\n|    BDTF | 72.27 | 66.12 | 61.74 | 74.35 |\n| TAGS(ours) | **76.61** | **67.90** | **64.53** | **75.05** |\n\nFrom the tabulated results, it becomes evident that our approach outperforms both of these newly introduced methods. Our model demonstrates superiority over these grid-based sequence labeling models, attributed not only to the nature of our generation task but also to the efficacy of our label-driven semantic alignment module. Our TAGS model notably outperforms the newly introduced comparative methods, providing further confirmation of the strengths of our paper and the effectiveness of our approach.\n\nFinally, it is important to note that the supplementary comparisons we have provided will not impact the experimental conclusions in our manuscript.\n\nBy incorporating these supplementary methods, we now present a more robust comparison of TAGS against a diverse set of baselines, showcasing its performance superiority and underscoring its unique contributions. We anticipate that these additional comparisons will enhance the overall quality of our work and its contributions to the ASTE research community.\n\n**For the reason to reject#3 \"The paper does not provide a detailed analysis of the limitations of the proposed TAGS model, such as its sensitivity to hyperparameters or its performance on datasets with different characteristics.\":**\n\nWe sincerely appreciate the reviewer's feedback and recognize the significance of providing a comprehensive analysis of the limitations of our proposed TAGS model. **Within the Limitations section of our paper, we have indeed outlined several limitations of our model, including concerns related to additional time overhead and the simplicity of the sequence tagging scheme.**\n\nIn our revised manuscript, we are committed to addressing this concern by offering a more thorough examination of the model's limitations.\n\n**Sensitivity to Hyperparameters:**\n\nOur primary hyperparameter configuration is defined as the generation loss ratio: tag loss ratio: align loss ratio = 10:1:1. Given that our task places a predominant emphasis on the generation task, the generation loss ratio is set relatively high. Our hyperparameter experiments systematically varied the generation loss ratio from 1 to 20, revealing that the optimal performance is achieved at a ratio of 10. In contrast, the other loss components serve as auxiliary constraints and are maintained at values near 1. Further hyperparameter experiments indicate that the ratios near 1 consistently yield the best results, and the results exhibit stable performance within the vicinity of 1.\n\nCrucially, our model's performance is not sensitive to hyperparameters. We have consistently employed the same set of hyperparameters across all four datasets and have consistently outperformed comparative methods using these configurations. The hyperparameter experiments detailed in the paper further validate the effectiveness of these chosen hyperparameters.\n\n**Performance on Datasets with Different Characteristics:**\n\nTo begin, it's important to highlight that the TAGS model exhibits significant improvements across all datasets, demonstrating its broad adaptability. \n\nHowever, it's worth noting that the extent of improvement with our TAGS model is more pronounced on more complex datasets. The simpler datasets are already well-handled by existing models. Due to their straightforward grammatical structures, these datasets pose fewer challenges in terms of erroneous triplet extractions, resulting in higher accuracy for existing models. Additionally, the simpler grammatical structures make correct triplet extractions relatively straightforward, leading to higher recall for existing models. In cases where the grammatical structure is simpler, such as the 14res dataset, our TAGS model doesn't show substantial improvement, as other models perform sufficiently well. \n\nIn contrast, our model bolsters the encoder's discriminative capabilities concerning triplet elements. Thus, even in complex datasets, the TAGS model enhances the extraction of implicit triplets, leading to improved recall. The tag attention mechanism guides the model to focus on keywords identified by the sequence tagging task, enabling it to avoid erroneous extractions of ambiguous triplets even in complex datasets. This mechanism enhances accuracy. Therefore, our TAGS model exhibits more substantial improvements on datasets with complex syntax, such as the 16res dataset.\n\nFurthermore, as detailed in our responses above (response to reject reason #1), we have conducted experiments on an additional dataset, specifically the 2023 ACL-released dataset. These datasets, generally more complex than the ones we currently use, provide a valuable testing ground for our TAGS model. Experimental results indicate that our TAGS model achieves significant and consistent improvements, with F1 scores increasing by 2.87% (Electronics), 2.8% (Beauty), 2.86% (Fashion), and 3.02% (Home) compared to the GAS model.\n\n**For question#1 \"What potential biases may affect the predicted sentiment polarities in this work? \":**\n\nFirst and foremost, it is imperative to emphasize that the design of our TAGS model intentionally avoids the introduction of any prior knowledge, aiming to minimize biases. Our model operates on the principles of natural language processing and machine learning and, fundamentally, does not favor any specific sentiment polarity.\n\nHowever, within the context of our work, it is worth acknowledging that pre-trained models may be exposed to biases in the pre-training data they utilize, potentially leading to biases in the predicted sentiment polarities. This is a challenge that most researchers encounter when utilizing pre-trained models. It is important to note that the issue of biases in pre-trained models has garnered significant attention. Notably, there has been a concerted effort towards curating more standardized training data for pretraining models, thereby mitigating the prevalence of biases. In our study, we employ the pre-trained generative model T5, developed by Google, which utilizes meticulously curated and cleaned datasets. This significantly reduces the potential bias inherited from the pre-training data.\n\nUnderstanding and rectifying biases is a critical undertaking in the domains of sentiment analysis and natural language processing. We are committed to promoting responsible AI practices and advancing the development of sentiment analysis systems that are more equitable and just.\n\n**For question#2 \"Does the article mention any limitations or challenges of the sequence tagging approach used in this work?\":**\n\nIndeed, it is noteworthy that the current sequence tagging approaches for the ASTE exhibit certain limitations, and the generation-based model we propose aims to address these limitations to a certain extent. **In our \"Limitations\" section, we acknowledge that our sequence tagging approach employs a simplified design, specifically characterized by the absence of explicit pairing between extracted aspects and opinions.** However, this limitation does not significantly impact the overall performance, as during the generation phase, the decoder leverages contextual information and comprehensively considers both the aspects and opinions outcomes from the sequence tagging task to combine and pair crucial elements within the sentences.\n\nThis deliberate simplification is a strategic choice made for the sequence tagging approach, which is designed for the generation model, and it is motivated by the following reasons:\n\n1) Firstly, to ensure that the generation task remains the central focus, we devised a streamlined sequence tagging scheme. Within this sequence tagging scheme and its associated loss functions, we deliberately omitted the explicit pairing of aspects and opinions. Consequently, this deliberate omission represents a relatively loose constraint. The rationale behind this choice lies in the potential adverse effects of imposing overly stringent constraints on the encoder, which could impede the learning process and the ultimate performance of the generation task. Striking an appropriate balance in constraint strength on the encoder allows us to enhance the encoder's discriminative capacity in identifying elements of the triplets, all the while benefiting the generation process.\n\n2) Furthermore, the design of this sequence tagging scheme enables the output results to be readily utilized by the tag attention component to guide the generation process.\n\n**For question#3 \"Does the article discuss any potential ethical concerns related to the use of large-scale pre-trained language models in sentiment analysis?\":**\n\nIn fact, our article does mention a range of ethical concerns associated with the use of large-scale pre-trained language models (LPMLs) in the broader domain of sentiment analysis. **In our Ethics Statement section, we explicitly state: \"Due to the nature of the Internet-based large-scale pre-training corpora, the predicted sentiment polarities may be influenced by unintended biases related to gender, race, and intersectional identities.\"** This statement underscores a critical issue, wherein LPMLs often inherit biases present in their training data, potentially leading to biased sentiment analysis results, particularly when assessing text from underrepresented or marginalized groups.\n\nFurthermore, in the revised version of our paper, we plan to delve more deeply into ethical considerations. Fortunately, these issues are actively being addressed within the research community, including efforts to standardize datasets and methodologies.\n\nSpecifically, within the scope of our task, as we primarily focus on product reviews, ethical considerations may not be as frequently encountered. Nonetheless, we remain vigilant about the broader ethical implications associated with LPMLs and sentiment analysis, and we are committed to contributing to responsible AI practices.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.The paper highlights the importance of considering biases in large-scale pre-trained language models and the need for more extensive consideration of biases in the NLP community.\n2.The creative aspects of this article include proposing a new approach to enhance the supervision of encoder and decoder hidden representations in Natural Language Generation-based methods for ASTE, integrating sequence tagging probabilities and semantic representations of labels\n3.The proposed Tagging-Assisted Generation Model with Encoder and Decoder Supervision (TAGS) enhances the supervision of encoder and decoder hidden representations in Natural Language Generation-based methods for Aspect Sentiment Triplet Extraction (ASTE), achieving enhanced semantic information utilization and improving the extraction of implicit aspects and opinions.", "reasons_to_reject": "1.The paper does not provide a detailed discussion of the potential practical applications of the proposed TAGS model, such as its scalability or its compatibility with different types of data.\n2. The paper does not compare the proposed TAGS model with a wider range of state-of-the-art methods for ASTE, which could provide a more comprehensive evaluation of its effectiveness.\n3.The paper does not provide a detailed analysis of the limitations of the proposed TAGS model, such as its sensitivity to hyperparameters or its performance on datasets with different characteristics.", "questions_to_authors": " 1.What potential biases may affect the predicted sentiment polarities in this work? \n 2.Does the article mention any limitations or challenges of the sequence tagging approach used in this work? \n 3.Does the article discuss any potential ethical concerns related to the use of large-scale pre-trained language models in sentiment analysis? ", "paper_topic_and_main_contributions": "This article proposes a new approach called Tagging-Assisted Generation Model with Encoder and Decoder Supervision (TAGS) for Aspect Sentiment Triplet Extraction (ASTE). The proposed model enhances the supervision of encoder and decoder hidden representations in Natural Language Generation-based methods for ASTE by integrating sequence tagging probabilities and semantic representations of labels. The model achieves enhanced semantic information utilization and improves the extraction of implicit aspects and opinions. The main contribution of this work is the proposed TAGS model, which outperforms the state-of-the-art methods on two benchmark datasets for ASTE. The work may contribute to further research in sentiment analysis and the use of generation methods to simplify and automate the extraction of user opinions in real-world applications.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ISaqgsGd1D", "reviewer": "Reviewer_Uz23", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our sincere gratitude to you for your valuable feedback and insights into our work.\n\n**For the reason to reject:**\n\nWe genuinely appreciate the reviewer's careful assessment of our work and would like to address the concern that our paper may be perceived as incremental. To clearly illustrate that our contribution extends beyond incremental work, we want to emphasize the following three key points:\n\n**1、Innovative Label Semantics Supervision:**\n\nIn our second contribution, we present a novel and creative attempt to enhance the supervision of the decoder in the generation model by utilizing the semantic representation of the labels. Specifically, we perform self-decoding on label sentences, where the decoder's output serves as the semantic representation of labels. This semantic representation is then used to supervise the content generated by the generation model. In essence, we parameterize label sentences to provide fine-grained, semantic-level labels. This is a shift from traditional one-hot label supervision to semantic label supervision. This approach brings forth various advantages, encompassing not only the benefits mentioned in the paper but also extending to capabilities such as disambiguating polysemous words—an intricate task that is not easily tackled through conventional one-hot labels. To the best of our knowledge, ours is the first work to adopt such a strategy. We believe that this novel approach holds significant promise and warrants attention.\n\n**2、The idea of choosing Sequence Tagging:**\nAfter successfully enhancing supervision over the decoder's hidden states with the semantic representation of labels, we realized that introducing an additional task to enhance supervision over the encoder's hidden states was also imperative. Generally, classification tasks put a strong supervision on the encoder. We believe that a classification-based sequence tagging task better strengthens the encoder's ability to distinguish between triplet and non-triplet elements. So, we have decided to include a sequence tagging task as an additional task to enhance supervision over the encoder.\n\nFurthermore, the sequence tagging task within our model is relatively straightforward. This deliberate design objectively demonstrates that the improvements in our model are not contingent upon the advantages brought by advanced sequence tagging methods.\n\n**3、Meticulously Designed Technical Details:**\n\nWe have innovatively and meticulously designed our work, rather than merely assembling existing works.\n\na) Firstly, to ensure that the generation task remains the central focus, we devised a streamlined sequence tagging scheme. Within this sequence tagging scheme and its associated loss functions, we deliberately omitted the explicit pairing of aspects and opinions. Consequently, this deliberate omission represents a relatively loose constraint. The rationale behind this choice lies in the potential adverse effects of imposing overly stringent constraints on the encoder, which could impede the learning process and the ultimate performance of the generation task. Striking an appropriate balance in constraint strength on the encoder allows us to enhance the encoder's discriminative capacity in identifying elements of the triplets, all the while benefiting the generation process.\n\nb) Furthermore, the design of this sequence tagging scheme enables the output results to be readily utilized by the tag attention component to guide the generation process.\n\nIn light of these contributions and distinctions from previously known algorithms, we hope that the reviewer will consider our work as an innovative and valuable contribution to the field of ASTE. We are fully committed to refining our manuscript to better communicate these aspects and to provide a compelling rationale for the innovative nature of our approach.\n\n**For the Missing References:**\n\nWe greatly appreciate your suggestion to include the work 'PASTE: A Tagging-Free Decoding Framework Using Pointer Networks for Aspect Sentiment Triplet Extraction' as a baseline in our paper. We acknowledge the significance of conducting comprehensive comparisons with existing methods to ensure a thorough evaluation of our contributions.\n\nAfter a meticulous review, we have identified that our work indeed shares some similarities with the 'PASTE' framework, as both are built upon the encoder-decoder architecture.\n\nIn our revised manuscript, we will incorporate \"PASTE\" as one of the key baseline methods, along with proper citations. The following Table 1 presents a comparison of our model's F1 performance with that of PASTE. The F1 performance of PASTE is copied from its respective publication.\n\nTable 1. Comparison with the missing reference.\n| Dataset    | 16res | 15res | 14lap | 14res |\n|------------|-------|-------|-------|-------|\n| PASTE      | 67.9  | 61.6  | 57.4  | 66.6  |\n| TAGS(ours) | **76.61** | **67.90** | **64.53** | **75.05** |\n\n\nThe TAGS model outperforms the PASTE model (e.g., over 8% improvement in the case of 16res). This disparity in performance can be attributed to the divergent objectives of the two models. While the PASTE model is primarily focused on predicting the position pointers of triple elements, TAGS, on the other hand, is designed to predict the natural language textual forms of the triplets. The TAGS approach, a generation-based method, is capable of leveraging the semantic information of the labels more comprehensively, thus yielding superior results. Furthermore, the TAGS model incorporates a label-driven semantic alignment module, which enables a more profound utilization of the semantic information encoded within the labels.\n\nFinally, it is important to note that the supplementary comparisons we have provided will not impact the experimental conclusions in our manuscript.\n\nThis addition will allow us to provide a more comprehensive and detailed comparative analysis, further enhancing the robustness and credibility of our research. We genuinely appreciate your valuable suggestion, and we are committed to ensuring that our revised paper reflects a comprehensive evaluation of our approach with relevant prior work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Authors proposed an effective combination of sequence labelling approach and generative approach to improve the performance of the generative models in ASTE task.\n2. Experimental results in four datasets show the effectiveness of their proposed approach on ASTE task, as well as on the ABSA and AOPE sub-tasks.\n3. Paper is well written and easy to follow. Authors have provided proper ablation studies and analysis to understand the proposed model.", "reasons_to_reject": "1. To some extent this is an incremental work that uses previously known algorithms and combine them to improve performance on the ASTE tasks.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposed a tagging-assisted encoder-decoder model for aspect-opinion-sentiment triplet extraction (ASTE). They add an additional sequence labelling task in an multi-task approach and use the label-sequence to improve the decoding process of the model. The sequence labelling component only finds the aspect terms and opinion terms along with the sentiment information. It does not pair the aspect terms with the opinion terms. The decoder module does the job of pairing them with the help of the sequence labels. They carry out experiments with four popular datasets in this task and achieved SOTA performance in four datasets. They also show the effectiveness of their model in various variation task such as AOPE, ABSA.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Following work is also based on encoder-decoder framework and should be cited and compared as a baseline.\n[1] PASTE: A Tagging-Free Decoding Framework Using Pointer Networks for Aspect Sentiment Triplet Extraction"}}]

Author rebuttal:

Meta-review: This paper proposes a new method for aspect sentiment triplet extraction. The authors propose a new approach to enhance the supervision of encoder and decoder's hidden representations in Natural Language Generation-based methods for ASTE, integrating sequence tagging probabilities and semantic representations of labels. This leads to new SOTA results. The minor points for improvement include more discussion of the application possibilities of the proposed methods, as well as a comparison to more prior methods, both of which have been provided in the author response.
Decision: Accept-Main