Target-to-Source Augmentation for Aspect Sentiment Triplet Extraction
Authors: Yice Zhang
yifan yang
Meng Li
Bin Liang
Shiwei Chen
Ruifeng Xu
Venue: EMNLP 2023 Main (2023)
Keywords: Aspect-Based Sentiment Analysis
Data Augmentation
Reinforcement Learning

Abstract: Aspect Sentiment Triplet Extraction (ASTE) is an important task in sentiment analysis, aiming to extract aspect-level opinions and sentiments from user-generated reviews. 
The fine-grained nature of ASTE incurs a high annotation cost, while the scarcity of annotated data limits the performance of existing methods.
This paper exploits data augmentation to address this issue.
Traditional augmentation methods typically modify the input sentences of existing samples via heuristic rules or language models, which have shown success in text classification tasks. 
However, applying these methods to fine-grained tasks like ASTE poses challenges in generating diverse augmented samples while maintaining alignment between modified sentences and origin labels.
Therefore, this paper proposes a target-to-source augmentation approach for ASTE.
Our approach focuses on learning a generator that can directly generate new sentences based on labels and syntactic templates. With this generator, we can generate a substantial number of diverse augmented samples by mixing labels and syntactic templates from different samples.
Besides, to ensure the quality of the generated sentence, we introduce fluency and alignment discriminators to provide feedback on the generated sentence and then use this feedback to optimize the generator via a reinforcement learning framework.
Experiments demonstrate that our approach significantly enhances the performance of existing ASTE models.

Target-to-Source Augmentation for Aspect Sentiment Triplet Extraction
Yice Zhang*,, Yifan Yang* Meng Li', Bin Liangl.244 1..44, Shiwei Chen Ruifeng Xu 1,23+
Harbin Insitute of Technology, Shenzhen, China
2 Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies
Peng Cheng Laboratory, Shenzhen, China The Chinese University of Hong Kong, Hong Kong, China (zhangyc_hite biit evany (e1. com
{
mimas 1i 11w.1117777aaa11 com, chenshw@pcl ac. cn
bin. Man.liang@culle edu. hk, xuruifeng@his edw.ehiittdudc......... cn
Abstract
Original Sentence and Label
The price is reasonable although the service is poor. ((price, reasonable, positive), (service, poor, negative))
Aspect Sentiment Triplet Extraction (ASTE) is an important task in sentiment amalysis, aiming extract aspect-level opinions sentiments from user-generated reviews. The fine- grained nature of ASTE incurs high annotation cost, while the scarcity of annotated data limits the performance of existing methods. This paper exploits data augmentation to address this is- sue. Traditional augmentation methods typi- cally modify the input sentences of existing samples heuristic rules or language models, which have shown success in text classifica- tion tasks. However, applying these methods to fine-grained tasks like ASTE poses challenges in generating diverse augmented samples while maintaining alignment between modified sen- tences and origin labels. Therefore, this paper proposes a larget-to-souree augmentation ap- proach for ASTE. Our approach focuses on learning generator that can directly generate new sentences based on labels and syntactic templates. With this generator, we can generate substantial number of diverse augmented sam- ples by mixing labels and syntactic templates from different samples. Besides, to ensure the quality of the generated sentence, we introduce fluency and alignment discriminators to pro- vide feedback on the generated sentence and then use this feedback to optimize the gener- ator via reinforcement learning framework. Experiments demonstrate that approach sig- nificantly enhances the performance of existing ASTE models.?
Traditional Augmentation modifies the original sentence and combines it with the original label to form augmented sample.
The price reasonable, but the service is poor. The price was reasonable although the service was poor.
Target-to-Souree Augmentation generates sentences based on labels and syntactic templates.
G label + pyntactic-template
The price was reasonable but service we received was lat least this time ).
label + cnttacti-teempaate2
If you in because the price is reasonable ( ($11) ) you will met with poor service.
Figure I: Samples synthesized by traditional augmenta- tion method and the proposed target-to- to-uuuce method.
natural language processing (Xu et al., 2020; Chen et al., 2021: Yan et al., 2021; Zhang et al., 2021; Xu et al., 2021; Chen et al., 2022; Zhang et al., 2022). The goal of ASTE is extract aspect-level opinions and sentiments from user-generated re- views. For example, given the review sentence "the price reasonable although the service poor". the output of ASTE would be ((rrice, reasonable, positive), (service, poor, negative)).
The ASTE task faces the challenge of data scarcity, and data augmentation has emerged as a potential solution to this issue. Annotating ASTE data is time-consuming and labor-intensive due to its fine-grained nature, leading to small exist- ing public ASTE datasets. Despite some few-shot attempts (Hosseini-Asl al., 2022; Liang et al., 2023), mainstream methods still require signifi- cant amount annotated data to achieve satisfac- tory performance. Data augmentation is the tech- nique that synthesizes new training samples based
1 Introduction
Aspect Sentiment Triplet Extraction (ASTE) is an important task in sentiment analysis (Peng et al., 2020), which is receiving increasing attention in
Corresponding Authors
'Wo release our code and data at https: //gi thub. wtt.:///ttnttttttaaaaaa Augmentation.

---

on existing datasets, and researchers have explored data augmentation methods to alleviate the prob- lem of data scarcity (Li et al., 2020; Wang et al., 2021; Hsu et al., 2021; Liang et al., 2021; Wang et al., 2022; Hu et al., 2022; Li et al., 2022).
the fluency discriminator and the alignment dis- criminator, to assess the quality of the generated sentences. Next, we employ a reinforcement learn- ing framework to optimize the generator based on the feedback from these two discriminators. Fi- nally, in the inference phase, we use these two discriminators to further filter out low-quality aug- mented samples. Experiments demonstrate that our augmentation approach outperforms previous aug- mentation methods and significantly improves the performance of existing ASTE models.
Traditional augmentation methods typically modify the input sentence of the existing sample and then combine the modified sentence with the original label to form an augmented sample (Zhang et al., 2015; Wei and Zou, 2019; Kobayashi, 2018; Wu et al., 2019). While such methods have shown improvements in text classification tasks, ensuring alignment between the modified sentence and the original label in fine-grained tasks like ASTE is challenging. Existing efforts often limit the range extent of modifications ensure alignment (Li al., 2020; Hsu et al., 2021; Gao al., 2022), but these approaches restrict the diversity of the augmented samples and thereby weaken the effec- tiveness of data augmentation. Furthermore, for the ASTE task, labels and syntactic structures are two primary sources of diversity. For example, con- sider the sentences "the price is reasonable" and "it provides an intimate setting". Firstly, their corre- sponding labels, (price, reasonable, positive) and (setting, intimate, positive), differ. Secondly, the syntactic relations also vary: reasonable and price exhibit an adjective complement (acomp) relation, whereas intimate and setring display an adjective modifier (amod) relation.
Related Work
2.1 Data Augmentation
Data augmentation aims at synthesizing new train- ing samples based existing datasets. Most data augmentation methods typically modify sentences through heuristic rules (Zhang et al., 2015; Wei and Zou, 2019) or language models (Kobayashi, 2018; Wu et al., 2019). The most representative bale- method is EDA (Wei and Zou, 2019), which involves four operations: synonym replace- ment, random insertion, random swap, and ran- dom deletion. The language model-based methods generally follow the rrroonssanndrooosstuuut frame- work, which first randomly masks words or spans in the sentence and then uses a language model fill the masked parts.
For fine- ane-grained tasks, these methods can easily lead to misalignment between the modified sen- tences and the original labels. There have been two main attempts to solve this problem:
Based on the above observations, this paper pro- poses target-to-source augmentation approach for ASTE. The main idea to learn a generator that can directly generate new sentences based on both labels and syntactic templates. Compared to traditional augmentation methods, our approach enables the generation of more diverse contexts for a given label by leveraging various syntactic templates. More importantly, we produce substantial number of augmented samples by com- bining labels and syntactic templates from differ- ent samples. This allows us to further select high- quality ones from these augmented samples to con- struct the augmented dataset.
Label-unrelated Modification: this method first identifies label-unrelated parts of the sen- tences through rules or selective perturbed masking and then only modifies these parts to avoid misaligmment (Li et al., 2020; Hsu et al., 2021; Gao et al., 2022).
2. Conditional Language Modeling: this method inputs both the label sequence and the sentence into the language model (Li et al., 2020) or linearizes the label into the sentence and inputs it into the language model (Ding al., 2020: Zhou et al., 2022).
In addition to diversity, augmented samples must also meet two requirements: fluency and alignment. Fluency refers to the ability of the generator pro- duce natural and coherent sentences. Alignment re- quires consistency between the given labels and the generated sentences. To satisfy these two require-
Although these attempts can appropriately alleviate the problem of misalignment, the issue still per- sists due to lack of supervision over the modified content.
In addition, there are other data augmentation methods such as tackstranslttion (Sennrich et al.,
ments, we introduce two discriminators, namely

---

2016), MixUp (Zhang et al., 2018), and reinforce- ment learning-guider generation (Liu al., 2020), but few works apply them to fine-grained tasks.
(ROOT)
price (nsubi)
reasonable (acomp)
(advel)
(punct)
Tree
2.2 Data Augmentation for Aspect-Based Sentiment Analysis
the (del)
although (mark) service (1sub) poor (acomp)
the (det)
Unlike sentence-level sentiment analysis, Aspect- Based Sentiment Analysis (ABSA) focuses on opinions and sentiments expressed on specific as- pects within a sentence (Pontiki et al., 2014). As- pect Sentiment Triplet Extraction (ASTE) is a rep- resentative task in the current research of ABSA, proposed by Peng et al. (2020).
Seq
ROOT [nsubj [det) acomp advel [mark nsubj (det) acomp] punct)
Figure 2: Prefix notation for dependency tree.
2. Alignment: the generated sentence should contain the given aspect-sentiment triplets and not introduce new aspect-level sentiment and opinion.
As fine-grained tasks, ABSA faces the problem of data scarcity, and researchers have developed various data augmentation methods to alleviate this issue. Dai and Song (2019) apply rules to label auxiliary data. Li et al. (2020) propose condi- tional data augmentation method for Aspect Term Extraction (ATE). Hsu al. (2021) adopt selective perturbed masking to select label-unrelated parts of sentences and then language models to re- place these parts. Wang et al. (2021) propose a pro- gressive self-training framework to infer pscudo- labels on the unlabeled data. Liang et al. (2021) design senect-iivariant-ddeeeeeee data augmenta- tion and deploy a supervised contrastive learning objective. Wang et al. (2022) conduct both aspect augmentation and polarity augmentation. Hu et al. (2022) propose template-orde order data augmentation method for generative ABSA.
the augmented sample should have significant differences from existing samples.
Diversity:
4 Proposed Framework
4.1 Model Architectures
To assist in the training of the generator, our frame- work includes two additional models: fluency discriminator and alignment discriminator. The fluency discriminator assesses the fluency of the generated sentence, and the alignment discrimina- tor evaluates whether the generated sentence aligns with aspect- triplets. These two models can be formulated as follows:
Dau (0,1),
(2)
3 Problem Definition
Daii : S L - (0.
(3)
Target-to-source augmentation aims to learn gen- erator that can generate corresponding sentences based on given labels and syntactic templates. In this paper, we use the dependency tree as the syn- tactic template. Formally, the generator is defined follows:
Our generator is a complete transformer structure, consisting of an encoder and decoder. The two discriminators consist of transformer encoder and binary classifier.
To the transformer model, we need to convert the aspect-sentiment triplets and dependency trees into flattened sequences. For the sepeect-entiment triplets, we use a template to convert each triplet into sequence and then concatenate multiple se- quences to form the label sequence (Zhang et al., 2021), which can be formulated as follows:
L
(1)
where I E L denotes the given label, i.e., a set of aspect-sentiment triplets, d E D denotes a de- pendency tree, and 8 C S denotes the generated sentence. The generated sentence and the given label together form augmented sample (s, I). We have three requirements for the augmented sample:
SeQiabel = a1 01 Pr; ... an On Pas (4)
where (a;, Oc, Ps) denotes aspect-sentiment triplet. For the dependency tree, we use prefix notation represent tree as a sequence (Lample and Charton, 2020), which illustrated in Figure
1. Fluency: the generated sentence should be close real-world sentences in terms of form and structure, without logical or grammatical errors.

---

4.2 Supervised Learning
place positive onee Lnsafee negetive
Dep
ROOT (nsub] (def) scomp conj (poby (det [oovmod]]ll punct)
Before applying reinforcement learning, it is im- portant to provide the generator with good ini- tialization. addition, our reinforcement learning framework also requires two discriminators to pro- vide feedback on the generated sentences. There- fore, prior to reinforcement learning, we train these three models using supervised learning.
Sentence The place
very unsale orea.
Lobel
service | Sreet I positive: food fasty positive
Dep
ROOT [onod ce conj punct)
Senfencee Great service and fusty food!
Generator
R Reinforcement Learning U pdate
Pseudo-labeled Dataset. The existing labeled dataset is too small to train usable generator and reliable discriminators. To overcome this limita- tion, we construct a pseudo-labeled dataset and train models on it. First, we train an ASTE model (Zhang et al., 2021) on the existing labeled dataset. Then, we use this model to generate pseudo-labels for the unlabeled data and select samples with high- confidence predictions? to construct the pseudo- labeled dataset D {(11, d1, (1,, ......).
Senercted Sentence: The service Great with
Reward Model
Label. service Great posiiivee Food positive
Fluency Discriminator Alignment Disciminator
Figure 3: The reinforcement learning framework for tuning a target-to-sonn generator.
fluent sentences:
8ij -Cl,,,d,),, ED.
(7)
Training Generator. We optimize the generator the pseudo-labeled dataset through the following loss function:
Dut Par- [ < flu(s,))) - O
(8)
i [1, i }}}.
where flu(-) is a fluency scorer' that gives score from 0 to for the fluency of sentence, and 6nu is fluency threshold, which we set to 0.1 based on empirical observations.
Ls Σ (GossEntropy((((:4)) (5) B.NdDED
Training Alignment Discriminator: Like the flu- ency discriminator, we input both aligned and non- aligned samples into the alignment discriminator and use the cross-entropy loss optimize it:
where denotes the size of D
Training Fluency Discriminator. We input both fluent and non-fluent sentences to the fluency dis- criminator and use the cross-entropy loss to opti- mize it:
LID.
Σ
log (atiss,
(9)
N
SAEDD
IDa.
1
Σ
log (8t)
(6)
Σ
log(1 Dati(s, (a hn)),
N1
N
$1
1
Σ 8nf
log(1 (stt),
where (s, la) and Ina) represent an aligned sam- ple and a non-aligned sample, respectively. To con- struct non-aligned samples, we use beam search to generate four prediction results on the trained ASTE model (Zhang et al., 2021) and then sample one of two predictions with lower confidence the non-aligned labels Ina-
2₂
where 8t and 8nf represent fluent sentence and non-fluent sentence, respectively, and N1 and N₂ are the total number of two categories.
We use the existing review sentences as fluent sentences, i.e., S1 E D. For non-fluent sentences, we generate them by mixing labels and dependency trees from different samples, inputting them into the trained generator generate sentences, and then considering the ones with low fluency non-
4.3 Reinforcement Learning
Supervised learning enables the generator to gen- erate sentences based on labels and dependency
com/ "This fluency scorer is from Saarrot__arappaasseeeeee trithiviral vvvirajuaamdaaran Parrot We don't directly use the result of this scorer to determine whether sentence is fluent or not because real review sentences also have probahility of heing scored low by the scorer.
"Specifically, we exclude pseudo-labels with confidence level below 0.75, i.c., those satisfying 3, P(x Me.. < 0.75. Refer to Appendix B.1 for the effect of the confidence threshold.

---

trees. However, when the label and dependency tree come from different samples, the generator tends to generate non-fluent or non-aligned sen- tence. To address this problem, we use reinforce- ment learning framework to train the generator. As shown in Figure 3, we employ two discriminators to provide feedback on the generated sentence and then optimize the generator based this feedback.
Dataset
#Sent
#Triplet
#Pos
#Neu
#Neg
Rest-14-train
1266
2338
1692
166
480
Rest- 14-dev
310
404
1estst4-test
492
994
773
66
155
Lap- 14-tran
906
1460
817
126
517
Tap- 14-dev
219
346
169
141
Lap-14-test
328
364
116
Yelp
427300
602394
400858
65388
136148
Amazon
188796
230265
129324
18573
82368
Reward Calculation. Given the label I and de- pendency tree dj. we denote the sentence generated by the generator as 8i5²
Statistics of four ASTE datasets (Xu al., 2020). #Sent and #Triplet represent the number of sen- tences and triplets, respectively. Besides, #Pos, #Neu, and #Neg represent the numbers of positive, neutral, and negative triplets, respectively.
Table 1:
Sij - RL (4, dj),
(10)
We input the generated sentence Sij into the fluency discriminator to obtain its fluency score finm. (8:5) and input both the generated sentence sij and the given label 4 into the alignment discriminator to ob- tain its alignment score SDAu (Sij. 4... In addition, to prevent the model from generating short and simple sentences without considering the dependency tree, we also introduce a length penalty peten (sij. dj).
generated sentence, we require them contain an equal number of triplets. Next, we input the label of the first sample and the dependency tree of the second sample into the generator GRL to generate the corresponding sentence. This process is repeated n1 times, resulting in nı augmented samples.
The reward is a combination of these three parts:
Subsequently. we employ the fluency discrimi- nator and the alignment discriminator filter out non-fluent or non-aligned samples. The remaining 12 samples will be used to build the augmented dataset.
r(sij. li, dj) = ffpnu (sig) fD (sij. 4)) peten (Sij. dz),
(11)
where the length penalty is calculated by:
10 (s, d) max(len - len(s),
Experiment
(12)
lcn(d)
5.1 Datasets
We evaluate the proposed augmentation method on destaurrnt-1 and haptop-14 of ASTE- D2ta-V2 (Xu et al., 2020). Accordingly, we uti- lize the Yelp Dataset* for Restaurant-14 and Amazon Review Dataset (Ni et al., 2019) for Lapt.pp-14 construct pseudo-labeled datasets. For each domain, utilize total of 10,000 re- views. We infer the aspect-sentiment triplets for each sentence of the reviews using a pre-trained ASTE model (Zhang et al., 2021) and retain only those sentences that contained triplets. The data statistics for these datasets are shown in Table 1.
Reinforcement Learning Update. Inspired by Ouyang et al. (2022), we use the Proximal Policy Optimization (PPO) algorithm (Schulman et al.. 2017) to perform reinforcement learning updates. We update the generator by maximizing the follow- ing objective function:
F(a.ld))DamL (iet,,)-Deearrr (r(s,
ÀKL KL d)),
where >KL is hyper-parameter which we empiri- cally set to 0.01, and the (,, is per-token KL penalty from the SL (supervised learning) gen- erator GSI, each token which is used mitigate over-optimization from the discriminators (Ouyang et al., 2022):
5.2 Implementation Details
We obtain the dependency tree for each sentence using spaCy Both the generator and the two dis-
(T, d) (R) (sl,,)//G
(s|1,
'https: /ttts://www... http://///wwwwwp......mm yelp. com/ dataset htttps://aijiaamen github io/ amazon/ index.html
4.4 Data Synthesis and Filtering
We randomly select two samples from the pscudo- labeled dataset δ. To guarantee the quality of the
"The trained pipeline we use is en en_core_wete 3.3.9.

---

Restaurant-14
Laptop-14
ASTE Model
Data Augmentation Method
P.
R.
Fi
A-Fi
P.
R.
Fi
A-Fi
No Augmentation
68.73
67.15
67.92
61.67
55.30
58.31
(Dii and Zou, 2019)
67.71
66.72
67.20
-0.72
55.10
50.69
52.79
-5.52
BARTABSA
Mask-then-Fülll(Gnn 2022)
65.48
63.58
64.50
-3.42
55.12
49.28
52.03
-6.28
(Yan et al., 2021)
Conditional-Augmeetaaiin ((Li et al., 2020)
70.20
67.98
69.06
+1.14
59.11
52.75
55.74
-2.57
Coudiiional-Auugmentttioo------- et al., 2020)
71.23
68,30
69.72
+1.80
59.41
54.53
56.84
-1.47
Target-to-Source Augmentation (Ours)
74.08
72.65
73,35
+5.43
63,84
59.15
61.39
+3.08
No Augmentation
73,50
71 .11
72.44
65.57
60.22
62.78
EDA(Wei and Zou, 2019)
70.70
70.24
70.47
-1.97
62.86
57.05
59.81
-2.97
GAS
Mask-thes then-FFil(GGao et al., 2022)
68.70
66.88
67.77
-4667
60.43
53.33
56.65
-6.13
(Zhang al., 2021)
(LLi et al., 2020)
72.50
70.81
71.64
-0.80
64.26
58.93
61.47
-1.31
Conditional Sattriioall-uuggmmattttoo------- et al., 2020)
73.58
70.65
72.08
-0.36
64.06
58.82
61.33
-1.45
Target-to-Source Augmentation (Ours)
75.65
74.57
75.11
+2.67
66.17
61.95
63.99
+1.21
No Augmentation
71.50
71.22
71,36
62.63
54.09
58.03
EDA(Wei and Zou. 2019)
71.02
66.50
68.63
-2.73
65.50
49.38
56.04
-1.99
Span-ASTE
Mask-then-Füll(Caae et al., 2022)
66.90
63.04
64.90
-6.46
53.77
46.36
49.78
-8.25
(Xu ct al., 2021)
CandiiioaalAuugmeetttti-o------- et al., 2020)
70.60
70.85
70.71
-0.65
60.20
53.09
56.36
-1.67
Canditional-uuggmnttttioo--LLL-- et al., 2020)
69.96
69.87
69.90
-1.46
60.56
53.94
57.04
-0.99
Target-to-Source Augmentation (Ours)
72.77
74.18
73.45
+2.09
62.99
60.04
61.46
+3.43
No Augmentation
76,29
72.43
74.30
65.72
56.41
60.66
EDA(Wei and Zou, 2019)
78.05
68.11
72.66
-1.64
70.85
51.13
59.29
-1.37
BDTF (Zhang al., 2022)
Mask Fill-oo et al., 2022)
73.73
61,55
67.02
-7.28
62,25
45.58
52.43
-8.23
Conditional Augiimmtal-ugmmettiooinn.. (IIi et al., 2020)
78.57
68.53
73.20
-1.10
71.39
54.13
60.23
-0.4
Conditional Sattriinaal-Auggmmtattiioooo 2(L. et al., 2020)
77.29
70.22
73.53
-0.77
69.59
52.35
59.62
-1.04
Torget-to-Soure Augmentation (Ours)
78.15
74.97
76.52
+2.22
67.48
58.11
62.43
+1.77
Table 2: Experimental results on ASTE-Data-v2 (Xu et al., 2020) (%).
criminators are initialized using T5- 15-large (Raf- fel et al., 2020). In the PPO algorithm, set X = 0.95 and 7 0.99. To harness its effec- tiveness, we employ several training strategies, in- cluding reward clipping, reward scaling, advan- tage normalization, and learning rate decay. For each domain, we generate 100,000 augmented sam- ples and select 5,000 samples based on alignment and fluency scores. These selected samples then merged with the previous dataset create an augmented dataset. We evaluate these augmented datasets using the existing ASTE models. To mini- mize the impact of randomness, we run each model 5 times with different random seeds and then report the average results.
Mask-then-Fill (Gao et al., 2022) first randomly masks out a sentence fragment and then infills variable-length text span with a fine-tuned infilling model. To preserve the sentence label, this method only masks uubellunrelated sentence fragments.
Conditional Augmentation (Li et al., 2020) is sim- ilar to mask-then-fill but with one key difference. Conditional augmentation additionally includes the label as sequence input to the infilling model. This method is proposed specifically for Aspect Term Extraction, where the label can be easily verted into a sequence of the same length as the input sentence. However, this method cannot be directly applied to the ASTE task, as its label is more fine-grained. We devise two solutions to ad- dress this issue: (1) conditionallaugmentttiio only takes the aspect and opinion label as the condi- tion; (2) conditionallaugmeenaan the triplets and the sentence together as input to the infilling model.
5.3 Baselines
Currently, there no data augmentation methods specifically designed for the ASTE task. Therefore, we select three data augmentation methods from similar tasks as baselines for comparison.
concatenates
Moreover, we select four representative ASTE models to evaluate the augmented datasets.
EDA (Wei and Zou, 2019) is classic rule-based augmentation method, which involves four oper- ations: synonym replacement, random insertion, random swap, and random deletion.
BARTABSA (Yan et al., 2021) and GAS (Zhang et al., 2021) are two generative approaches.

---

BARTABSA transforms ASTE into index gener- ation problem and utilizes the BART (Lewis et al., 2020) model to address it. GAS transforms ASTE into a text generation problem and employs the T5 (Raffel et al., 2020) model to solve it.
Discriminator Score
Restaurant-14
Laptop-14
high
90.00
86.67
Fluency
medium
65.00
52.50
low 83.33
73.33
high
86.67
80.00
Aligmment
medium
65.00
62.50
Span-ASTE (Xu al., 2021) performs term extrac- tion and relation classification through the shared span representations. Additionally, this approach introduces a dual-channel span pruning strategy to mitigate the high computational cost caused by span enumeration.
low
100.00
86.67
Table 3: Human evaluation of discriminators (accuracy, %).
Derformance on 11stauront-14
Derformance on Bestaurant-
15
BDTF (Zhang et al., 2022) represents each triplet as relation region in the 2D table and transforms the ASTE task into the detection and classification of these relation regions.
76
72
70-
70
5.4 Main Results
50-
60
Table lists the comparison results between the proposed augmentation approach and the previous augmentation methods. It can be observed that the previous methods have failed on the ASTE task. Instead of improving performance, they have re- sulted in varying degrees of performance degrada- tion. Particularly, the mask-then-fill method has led performance drop of over 3%. Despite incor- porating labels as conditions, the conditional aug- mentation method still fails to achieve performance improvements most cases. These observations indicate that ensuring alignment between the modi- fied sentences and the original labels is challenging for fine- fine-grained tasks like ASTE. Without explicit supervision of alignment, models are prone to gen- erating misaligned augmented samples, which will ultimately harm the model's performance.
66
06
00
Alig mmmnt Scores flowe high)
Fluency Scores (low > high)
Figure 4: Performance of GAS on the augmented dataset under different alignment and fluency scores (F) -corre %)
human annotator labels each sentence as fluent or non-f con-fluent. We exploit the same method to evalu- ate the alignment discriminator. As shown in Table 3, these two discriminators generally align with human judgment.
5.6 Further Analysis
In contrast, our approach consistently achieves improvements on both datasets, demonstrating the effectiveness of our approach. Compared to hav- ing no augmentation, our approach yields per- formance improvements ranging from 1.21% % to 5.43%. These improvements are particularly evi- dent for models that initially have poorer perfor- mance.
Effect of Alignment and Fluency. We conduct further experiments to analyze the impact of align- ment and fluency on the performance of the aug- mented dataset. From the generated large pool of augmented samples, perform sample selec- tion based on alignment and fluency scores, rang- ing from low to high. Subsequently, we evaluate the performance of the corresponding augmented datasets and present the results Figure 4.
5.5 Evaluation of Discriminators
These results indicate that both alignment scores and fluency scores have a positive impact on the performance of the augmented dataset, with higher scores generally leading better performance. Among these two scores, the alignment scores have particularly significant influence, while the im- pact of fluency scores is relatively mild. This find- ing is somewhat unexpected, suggesting that al-
We conduct human evaluations on small data batches to evaluate the fluency and aligmment dis- criminators. We choose 100 samples from the aug- mented samples synthesized by the generator. Of these, 50 are classified as fluent by the fluency dis- criminator, and the remaining 50 are classified as non con-fluent. These samples are then shuffled, and a

---

78.0
drop as well. This highlights the indispensability of these components in the augmentation process. In addition, we explore the possibility of using the pseudo-labeled dataset as a substitute for the generated augmented samples. The experimental results in Table 4 demonstrate the potential of this approach. Clearly, the pseudo-labeled dataset itself can bring greater diversity. However, the quality of pseudo-labels remains a major concern.
Peformance on Restourants
92.0
Laptop
75.0
51.5
74.5
65.0
24.0
Performa
12.0
61.6
72.5
61.0
12.0
67.5
2000
1000
6000
8000
Augmented Samples
Figure 5:
Performance of GAS on the augmented
datasets under different numbers of augmented sam- ples (Pi-score, %)
5.7
Case Study
Table 5 presents several representative examples to provide an intuitive understanding of the proposed augmentation approach. Observe that the mask- then-fill method easily introduces additional aspect terms opinion terms, leading to misaligned sam- ples. In comparison, this issue is less severe in con- ditional augmentation. Furthermore, both methods exhibit limited modifications the original sen- tences, resulting in a lack of diversity in the mod- ified sentences. Compared to these methods, our approach generally generates sentences with higher alignment and diversity. Moreover, as our approach can generate numerous sentences by introducing different syntactic templates, even if low-aligned samples are generated, we can easily filter them out using alignment scores. Such an advantage ensures the quality of the augmented dataset.
Dataset
Restaurant-14
Laptop-14
No Augmentation
72.44
62.78
Self-training
74.47
62.69
Target-to-Source
75.11
63,99
w/o reinforcement learning
74.24
60.70
w/o label&
73.39
62.31
Table 4: Ablation study (Fi-score %).. Self-training refers to using pseudo-labeled data augmented sam- ples.
though non-fluent sentences may be unfriendly to human readers, their influence on model training is limited.
Effect of Quantity of Augmented Samples. The quantity of augmented samples also crucial factor the effectiveness of data augmentation. Increasing the number of augmented samples pro- vides the model with more diverse range of con- texts, thereby enhancing the model's generalization ability. The results depicted in Figure 5 confirm the above statement. As the data quantity increases, the model's performance gradually improves. How- ever, once the data quantity exceeds 3000, the ad- ditional improvements become marginal. This ob- servation suggests that the diversity introduced by augmented samples still limited. Samples with high alignment scores often exhibit relatively nar- row dependency between aspect and opinion, while more complex dependencies can result in looser alignment. Further exploration of this challenge is warranted in further research.
Conclusion
In this paper, we propose target-to-souree aug- mentation approach to alleviate the issue of data scarcity in Aspect Sentiment Triplet Extraction (ASTE). Unlike traditional augmentation methods that modify input sentences, the target-to-source approach focuses on learning generator to di- rectly generate new sentences based on labels and syntactic templates. By combining the labels and syntactic templates from different samples, this generator can generate substantial number of di- verse augmented samples. To train the generator, we employ a reinforcement learning framework. Specifically, we introduce a fluency discriminator and an alignment discriminator to provide feed- back for the generated sentences. Then, we utilize the Proximal Policy Optimization (PPO) algorithm to optimize the generator based on this feedback. We apply the proposed augmentation approach public ASTE datasets and evaluate the augmented datasets using four representative ASTE models. The experimental results show that the proposed
Ablation Study. We conduct ablation experiments to analyze the effectiveness of each module. As shown in Table 4, removing reinforcement learn- ing results in significant performance decrease. Furthermore, when generating augmented samples using only the labels and syntactic templates from the original training set, we observe a performance

---

Augmentation Methods
Review Sentence
Alignment Scores
Original Sentence
The place is so cool and service prompt and curtious. The place clean and cool and the service prompt and curtious.
Mask-then-Fill
-2.66
Conditional Augmentation
The place very cool and the service prompt and curtious. In the end we chose cool place w/ prompt and curtious service.
4.63
Target-to Source Augmentation (1)
3.73
The place is very cool, the service staff was prompt, and the service was
Target-to-Sorce Augmentation
-0.31
curtious, looked forward to ordering our food after our meal.
Original Sentence
The menu is very limited i think we counted 4 or entrees.
Mask-then-Fill
The menu is very limited and expensive.
-2.59
Conditional Augmentation
The menu is very limited you can only get couple of the main entrees.
4.25
arrived at 7:30 and found myself. with such limited menu the corner,
Target-to-Soune Augmentation (I)
4.34
I had to do a double take.
First off. if you have a craving for scotch eggs, this is the place for you
Target-to-Souree Augmentation
1.72
but be aware that the menu is limited.
Unfortunately, the food is outstanding, but everything else about this restaurant the pits.
Original Sentence
Mask-then-Fill
Unfortuatelly the food is outstanding and delicious... Unfortunately the food is outstanding, but the service is bit of pits.
-2.59
Conditional Augmentation
-1.63
Target-to-Soune Augmentation (1)
The food at Elmer's 's is outstanding.
3.84
We came for outstanding food so if you ever visit Poriland, you should
Target-to-Souree Augmentation (2)
5.44
consider heading Sava.
Table 5: Case Study. The aspect terms and the opinion terms are marked with blue and red, respectively. Mask-then- and Conditional Augmentation, the modified parts marked using underlines.
augmentation approach significantly outperforms the previous augmentation methods and greatly im- proves the performance of existing ASTE models, demonstrating its effectiveness.
inevitably introduce noise. This noise can have a negative impact on data augmentation.
Our approach achieves diversity by introduc- ing different syntactic templates for each la- bel. However, when the syntactic template does not match the given label, the genera- tor, even after reinforcement learning tuning, tends to produce significant proportion of incoherent and hard-to-understand sentences. Although we can filter these out, we still have to generate large number of samples for fil- tering. Future work can explore how to better introduce diversity.
Acknowledgments
We thank the anonymous reviewers for their valuable suggestions to improve the quality of this work. This work was partially supported by the National Natural Science Foundation of China (62006062, 62176076), Natural Science Foundation of GuangDong Key Technologies Research and Development Program of Shenzhen 180G2000000214440000 Shenzhen Foundational Research Funding JCY1202208 000002008880041103330 Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies 002211212010005
Augmented samples inevitably contain noises. Therefore, to better utilize the augmented samples, it is necessary to develop a noise- insensitive training method, which is not ad- dressed in this paper. This is an area that can be explored in future research.
Limitations
Although our approach significantly improves the performance of existing ASTE models, it also suf- fers from the following limitations:
We believe that addressing the above limitations can further enhance the development of data aug- mentation techniques.
Our approach relies on two discriminators to evaluate the quality of the augmented sam- ples. However, these two discriminators are trained on pseude-labeled datasets, which will

---

References
Sosuke Kobayashi. 2018. Contextual augmentation: Data augmentation by words with paradigmatic re- lations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume 2 (Short Papers), pages 452-457, New Orleans, Louisiana. Association for Computa- tional Linguistics.
Hao Chen, Zepeng Zhai, Fangxiang Feng, Ruifan Li. and Xiaojie Wang. 2022. Enhanced multi- channel graph convolutional network for aspect sentiment triplct extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 2974-2985, Dublin, Ireland. Association for Computational Lin- guistics.
Guillaume Lample and François Charton. 2020. Deep learning for symbolic mathematics. In International Conference on Learning Representations.
Shaowei Chen, Yu Wang, Jie Liu, and Yuelin Wang. 2021. Bidirectional machine reading comprehension for aspect sentiment triplet extraction. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14):12666-1261
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sen se-suueeec pre- pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880, Online, Association for Computa- tional Linguistics.
Hongliang Dai and Yangqiu Song. 2019. Neural aspect and opinion term extraction with mined rules as weak supervision. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 1266,, Florence, Italy. Association for Computational Linguistics.
Guang-Dong Li, Hongya Wang. Yigiong Ding, Kan- gan Zhou, and Xiaowei Yan. 2022. Data augmen- tation for aspect- -aaect sentiment analysis. Interna- tional Journal of Machine Learning and Cyhernetics, 14:125-133 133.
Bosheng Ding, Linlin Liu, Lidong Bing, Canasai Kru- engkrai, Thien Hai Nguyen, Shafiq Joty, Luo Si, and Chunyan Miao. 2020. DAGA: Data augmentation with a generation approach for low-resouree tagging tasks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6045-6057, Online, Association for Computational Linguistics.
Kun Li, Chengbo Chen, Xiaojun Quan. Qing Ling, and Yan Song. 2020. Conditional augmentation for aspect term extraction via masked sequence-to- sequence generation. Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 7056-7066, Online. Association for Computational Linguistics.
Jun Gao, Changlong Yu. Wei Wang, Huan Zhao, and Ruifeng Xu. 2022. Mask-then-fill: A flexible and effective data augmentation framework for event ex- traction. In Findings of the Association for Computa- tional Linguistics: EMNLP 2022, pages 4537-4544 Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Bin Liang, Xiang Li, Lin Gui, Yonghao Fu, Yulan He, Min Yang, and Ruifeng Xu. 2023. Few-shot aspect category sentiment analysis via meta-learning, ACM Trans. Inf. Syst. 41(1).
Bin Liang, Wangda Luo, Xiang Li, Lin Gui, Min Yang, Xiaoqi Yu, and Ruifeng Xu. 2021. Enhanc- ing Aspect- Sentiment Analysis with Supervised Contrastive Learning, page 3242-3247 Association for Computing Machinery, New York, NY, USA.
Ehsan Hosseini-Ase Asl, Wenhao Liu, and Caiming Xiong, 2022. A generative language model for few-shot aspect-based sentiment analysis. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 770-787, Scattle, United States. Associ- ation for Computational Linguistics.
Ruibo Liu, Guangxuan Xu, Chenyan Jia, Weicheng Ma, Lili Wang, and Soroush Vosoughi. 2020. Data boost: Text data augmentation through reinforcement learning guided conditional generation. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9031-9041, Online. Association for Computational Linguistics.
Ting- Wei Hsu, Chung-CC Ching-Chiie Chen, Hen-Hsen Huang, and Hsi Chen. 2021. Semmantcs--reserveed data augmentation for aspect-based sentiment analysis. In Proceedings of the 2021 Conference on Empiri- cal Methods in Natural Language Processing, pages 4417-4422, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics.
Jianmo Ni. Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distently-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th Interna- tional Joint Conference on Natural Language Pro- cessing (EMNLP-IJCNLP), pages 188-197, Hong Kong, China. Association for Computational Lin- guistics.
Mengting Hu, Yike Wu, Hang Gao, Yinhao Bai, and Shiwan Zhao. 2022. Improving aspect sentiment quad prediction via template-order data augmenta- tion. In Proceedings of the 2022 Conference on Em- pirical Methods in Natural Language Processing, pages 7889-7900, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

---

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida. Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages Curran Associates, Inc.
Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP- IJCNIP), pages 6382-6388, Hong Kong, China. As- sociation for Computational Linguistics.
Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. Conditional bert contextual augmentation. In Computational Science ICCS 2019, pages 84-95, Cham. Springer International Publishing.
Haiyun Peng, Lu Xu, Lidong Bing, Fei Huang, Wei Lu, and Luo Si. 2020. Knowing what, how and why: A near complete solution for aspect-based sentiment analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8600-88)) -6077
Lu Xu, Yew Ken Chia, and Lidong Bing. 2021. Learn- ing span-level interactions for aspect sentiment triplet extraction. In Proceedings of the 59th Anmual Meet- ing of the Association for Computational Linguistics and the 1 Iith International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers). pages 4755-4766, Online. Association for Computa- tional Linguistics.
Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Har- ris Papagcorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2011 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (Se- mEval 2014), pages 27-35, Dublin, Ireland. Associa- tion for Computational Linguistics.
Lu Xu, Hao Li, Wei Lu, and Lidong Bing. 2020. Position aware tagging for aspect sentiment triplet extraction, In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2339-233220 2349, Online, Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn, Res., 21(1).
Hang Yan. Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, and Xipeng Qiu. 2021. A unified generative framework for various NER subtasks. of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing ( (Voiume I: Long Papers), ), pages 5808 5808-5822, Online. Association for Computational Linguisties.
Proceedings
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 86-96. Berlin, Germany, Association for Computational Lin- guistics.
Hongyi Zhang, Moustapha Cisse. Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations.
Wenxuan Zhang, Xin Li, Yang Deng, Lidong Bing, and Wai Lam. 2021. Towards generative aspect-based sentiment analysis. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguisties and the 11th International Joint Confer- ence on Natural Language Processing ( (Volume 2: Short Papers), pages 504-510, Online. Association for Computational Linguistics.
Bing Wang, Liang Ding, Qihuang Zhong, Ximing Li, and Dacheng Tao. 2022. A contrastive cross- channel data augmentation framework for aspect- based sen- timent analysis. In Proceedings of the 29th Inter- national Conference on Computational Linguistics. pages 6691-6704, Gyeongju. Republic of Korea. In- ternational Committee on Computational Linguistics.
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In Advances in Neural Information Pro- cessing Systems, volume 28, Curran Associates, Inc.
Qianlong Wang, Zhiyuan Wen, Qin Zhao, Min Yang, and Ruifeng Xu. 2021. Progressive self-training with discriminator for aspect term extraction. In Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 257-268, Online and Punta Cana. Dominican Republic. Asso- ciation for Computational Linguistics.
Yice Zhang, Yifan Yang, Yihui Li, Bin Liang, Shiwei Chen, Yixue Dang, Min Yang, and Ruifeng Xu. 2022. Boundary driven table-filling for aspect sentiment triplet extraction. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 6485-6498, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Jason Wei and Kai Zou. 2019. EDA: Fasy data augmen- tation techniques for boosting performance text classification tasks. In Proceedings of the 2019 Con- ference on Empirical Methods in Natural Language

---

Ran Zhou, Xin Li, Ruidan He, Lidong Bing, Erik Cam- bria, Luo Si, and Chunyan Miao. 2022. MELM: Data augmentation with masked entity language model- ing for low-resource NER. In Proceedings the 60th Annual Meeting of the Association for Compu- lational Linguisticy (Volume 1: Long Pupers), pages 2251-2262, Dublin, Ireland. Association for Compu- tational Linguistics.
89%. While perfect, the discriminator can identify misalignment in the majority of cases. Importantly, our observations indicate that while borderline cases might exhibit some misalignment, samples with higher alignment scores rarely demonstrate such issues.
In summary, we employ filtering strategies to mit- igate noise impact, effectively reducing noise to an acceptable level. Furthermore, the influence of noise on our framework remains limited.
A Discussion on Augmentation Using ChatGPT
As shown Table 7, utilizing ChatGPT to generate sentences based on labels and dependency trees is feasible, and GPT-4 has demonstrated promising capabilities in this regard. For simpler syntactic structures, GPT-4 can generate sentences that ad- here the requirements. However, when faced with more complex syntactic structures, the out- put from GPT-4 becomes less predictable and may easily yield misalignment with the given labels.
B.1 Effect of Confidence Threshold
Confidence Threshold
Alignment Accuracy
88.20
88.73
0.9
88.03
Table Effect confidence threshold on the perfor- mance of the alignment discriminator on Restaurant- 14 (aceuracy, %). We evaluate the performance of the alignment discriminator based its performance the original training set.
B Discussion on Noise
Pseudo-labeling inevitably introduces noise, which is indeed noteworthy limitation our approach. To mitigate the potential influence of this noise, we have devised the following strategies:
We have discovered that variations the confi- dence threshold yield relatively marginal impact on the alignment discriminator, as evidenced by Table 6. Furthermore, we can reasonably infer that the influence of this confidence threshold subse- quent reinforcement learning tuning also limited.
We remove samples with invalid labels, in- cluding aspect terms and opinion terms not present in the sentence, along with label se- quences that are structurally invalid.
We implement further filtration based on the model's own confidence scores to exclude la- bels with low confidence.
Based observations, these two steps collec- tively contribute significant reduction in noise levels.
Subsequently, we discuss the effects of noise on the generator and alignment discriminator.
Concerning the generator, it's important to un- derscore that supervised training with pseudo- labels merely equips it with preliminary sen- tence generation capabilities. We refine label- sentence aligmment through the subsequent reinforcement learning tuning, which heavily relies on the performance of the alignment discriminator.
Turning the discriminator, we evaluate its performance on the original training set and find achieves an impressive accuracy of

---

Label
non-tasty negative
Dependency Tree
ROOT [det prep [pohi] ce conj [der prep [pobil] prep [pobi [nmod]] punct]
GPT-4
The dish was non-tasty with bland flavors and lacked any appealing presentation. non-tasty dish and lot of for $80.
Our Approach
Label
server jerk negative
Dependency Tree
ROOT [nsubj advmod dobj [det] ec conj [aux xcomp [advmod]] punct punct punct]
GPT-4
The
jerk, even trying.
Our Approuch
I just had jerk server and will be staying away!!!
Label
staff' unprofessional negative; staff I shady/ negative
Dependency Tree
RÓOT [prep (prep spobj [det compound) ce conj [pobj [nmod [det nmod prep [pobjii compound
[amod]]]]] nsubj [det] aux aux att [neg amod quantmod quantmod] punct]
GPT-4
Amidst the unprofessional hehavior of some staff members, the atmosphere turned shady and the
overall experience was not pleasant.
Our Approach
Based the unprofessional and shady staff the Orlando Inn Fashion high end resort this place should have been no more than 4 stars.
Label
service negative: manager unprofessional inguttive
Dependency Tree
ROOT [nsubj relel [nsubj advmod prep [pohi]]] acomp [advmod] conj [nsuhjpass [det pound] auxpass] punct)
GPT-4
The service, whose manager was unprofessional, was bad and the staff; including the manager, were criticized.
Our Approach
The service received here ut tch restaurant was really bad and the manager was assigned was unprofessional.
Table 7: Several examples of using GPT-4 for target-to-source augmentation.


Reviews:
[{"review_id": "243UOBNHaP", "reviewer": "Reviewer_DtPr", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our great appreciation for your valuable comments to improve the quality of this manuscript.\n\n**Q1: How to mitigate the influence of noise from pseudo-labels on the generator and alignment discriminator?**\n\nA1: Pseudo-labeling inevitably introduces noise, which is indeed a noteworthy limitation of our approach. To mitigate the potential influence of this noise, we have devised the following strategies:\n\n-\tWe remove samples with invalid labels, including aspect terms and opinion terms not present in the sentence, along with label sequences that are structurally invalid.\n-\tWe implement further filtration based on the model's own confidence scores to exclude labels with low confidence.\n\nBased on our observations, these two steps collectively contribute to a significant reduction in noise levels. Subsequently, we discuss the effects of noise on the generator and alignment discriminator.\n\n-\tConcerning the generator, it’s important to underscore that supervised training with pseudo-labels merely equips it with preliminary sentence generation capabilities. We refine label-sentence alignment through the subsequent reinforcement learning tuning, which heavily relies on the performance of the alignment discriminator.\n-\tTurning to the discriminator, we evaluate its performance on the original training set and find it achieves an impressive accuracy of 89%. While not perfect, the discriminator can identify misalignment in the majority of cases. Importantly, our observations indicate that while borderline cases might exhibit some misalignment, samples with higher alignment scores rarely demonstrate such issues.\n\nIn summary, we employ filtering strategies to mitigate noise impact, effectively reducing noise to an acceptable level. Furthermore, the influence of noise on our framework remains limited. A comprehensive analysis of noise will be incorporated into this manuscript, and we deeply appreciate your valuable comment.\n\n**Q2: How to make the PPO algorithm effective?**\n\nA2: Undoubtedly, achieving effectiveness with the PPO algorithm is not easy due to its training intricacies and inherent instability. To harness its effectiveness, we have employed several training strategies, including reward clipping, reward scaling, advantage normalization, and learning rate decay. Furthermore, extensive hyperparameter tuning has been conducted, involving parameters such as reward clip threshold, learning rate, kl_coef, vf_coef, lambda, and gamma. We appreciate your insights, and we will incorporate these details into the manuscript to enhance the reproducibility of our work.\n\n**Q3: Is it feasible to generate samples using ChatGPT?**\n\nA3: Utilizing ChatGPT to generate sentences based on labels and dependency trees is feasible, and GPT-4 has demonstrated promising capabilities in this regard. For simpler syntactic structures, GPT-4 can generate sentences that adhere to the requirements. However, when faced with more complex syntactic structures, the output from GPT-4 becomes less predictable and may easily yield misalignment with the given labels. Below are a few examples.\n\nExample1: \n\n-\t**Label**: dish | non-tasty | negative\n-\t**Dependency tree**: ROOT [det prep [pobj] cc conj [det prep [pobj]] prep [pobj [nmod]] punct]\n-\t**GPT-4**: The dish was non-tasty with bland flavors and lacked any appealing presentation.\n-\t**Our approach**: A non-tasty dish and a lot of it for $80.\n\nEample2:\n\n-\t**Label**: server | jerk | negative\n-\t**Dependency tree**: ROOT [nsubj advmod dobj [det] cc conj [aux xcomp [advmod]] punct punct punct]\n-\t**GPT-4**: The server was a jerk, not even trying.\n-\t**Our approach**: I just had a jerk server and will be staying away!!!\n\nEample3:\n\n-\t**Label**: staff | unprofessional | negative ; staff | shady | negative\n-\t**Dependency tree**: ROOT [prep [prep [pobj [det compound] cc conj [pobj [nmod [det nmod prep [pobj]] compound [amod]]]]] nsubj [det] aux aux attr [neg amod quantmod quantmod] punct]\n-\t**GPT-4**: Amidst the unprofessional behavior of some staff members, the atmosphere turned shady and the overall experience was not at all pleasant.\n-\t**Our approach**: Based on the unprofessional and shady staff at the Orlando Inn of Fashion high end resort this place should have been no more than 4 stars.\n\nExample4:\n\n-\t**Label**: service | bad | negative ; manager | unprofessional | negative\n-\t**Dependency tree**: ROOT [nsubj [det relcl [nsubj advmod prep [pobj]]] acomp [advmod] cc conj [nsubjpass [det compound] auxpass] punct]\n-\t**GPT-4**: The service, whose manager was unprofessional, was bad and the staff, including the manager, were criticized.\n-\t**Our approach**: The service we received here at teh restaurant was really bad and the manager I was assigned was unprofessional.\n\nFinally, we would like to extend my sincere gratitude for your insightful and constructive feedback once again. Your input has been instrumental in refining our work, and we greatly appreciate your thoughtful engagement.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The techniques sound well, like the usages of fluency discriminator and alignment discriminator;\n2. In principle, the syntactic tree may boost generation diversity.\n3. The paper writing is clear;", "reasons_to_reject": "1. The accuracy of the pseudo labels may bring in some noise, how to balance the benefit and noise?\n2. The discriminators have the same problem, that is the low accuracy may introduce more noise, how to mitigate this?\n3. The PPO algorithm is not so easy to be effective, how do you achieve it?\n4. The performance of the ppo algorithm is affected by the performance of the discriminators a lot. What kind of performance of the discriminators can make PPO effective?", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposed a data augmentation method for ASTE. \n1) Their method is fine-grained, which would not reverse the sentiment label due to the alignment discriminator;\n2) Their method can generate fluency text based on a fluency discriminator;\n3) The performance of their method is superior.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Can the syntactic information be proceeded by chatgpt and output the corresponding response? I'm just curious, it doesn't affect the rating"}, "extra_text": {}}, {"review_id": "2U7QxI7NJJ", "reviewer": "Reviewer_Xree", "summary": "", "questions": "1. The alignment discriminator is trained by composing aligned and disaligned examples based on beam search and confidence scores. I wonder what is the beam size and the threshold confidence scores for selecting the negative samples. And how do you decide on these values? \n2. Fig 5 shows the trend by changing number of augmentations. What about changing the confidence thresholds?\n3. Fig 4 indicates the trends with both scores. Despite the scores given by two models, how about the actual fluency and alignment scores? Even a small set with human-evaluated scores on fluency and alignment is beneficial to see whether the discriminators actually align with humans' perceptions.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your invaluable feedback aimed at enhancing the quality of this manuscript.\n\n**Q1: Details of data construction for training the alignment discriminator.**\n\nA1: Thank you for your valuable feedback. We apologize for any lack of clarity in our manuscript and are committed to refining it for improved comprehensibility.\n\nWe train our generator and discriminators on a pseudo-labeled dataset. To ensure the reliability of these pseudo-labels, we retain only those pseudo-labeled samples with high confidence. Specifically, we exclude labels with *minimum-confidence* below 0.75, i.e., those satisfying $\\min P(y_t|y_{<t},x)<0.75$. Here, we empirically choose 0.75 as the confidence threshold, and we observe that our approach is not particularly sensitive to this threshold, as illustrated in *A2*.\n\nIn training of the alignment discriminator, for a given sample $(x,y)$ in the pseudo-labeled dataset, we utilize the ASTE model with beam search to generate four pseudo-labels, ordered by confidence as $(y_1,y_2,y_3,y_4)$, where typically $y_1=y$. We treat $(x,y_1)$ as the aligned sample, i.e., the positive sample; we treat $(x,y_3)$ and $(x,y_4)$ as non-aligned samples, i.e., negative samples. In the above process, we set the beam size to 4, because a larger beam size would generate obviously unreasonable negative samples; we do not treat $y_2$ as a negative sample to reduce potential noise.\n\n\n**Q2: The impact of the confidence threshold.**\n\nA2: We extend our appreciation for your insightful comments. We have discovered that variations in the confidence threshold yield a relatively marginal impact on the alignment discriminator, as evidenced by the following table. Furthermore, we can reasonably infer that the influence of this confidence threshold on subsequent reinforcement learning tuning is also limited. We thank you for highlighting this aspect, and we will incorporate this analysis to further enhance the manuscript.\n\n| Confidence Threshold | Alignment Accuracy in Training Set |\n| :---------------: | :------: | \n| 0.6 | 0.8820 | \n| 0.7 | 0.8820 |\n| 0.8 | 0.8873 | \n| 0.9 | 0.8803 |\n\n**Q3: Human evaluation for the fluency and alignment discriminators**\n\nA3: Thank you for your constructive suggestions. Given the critical role of these two discriminators in our approach, we agree on the necessity of subjecting them to human evaluation. We will incorporate this into the manuscript.\n\nFinally, we want to convey my sincere gratitude for your insightful and constructive feedback once again. Your contributions have been instrumental in the enhancement of our work, and we deeply value your thoughtful engagement.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea of automatically training a generator for augmenting the ASTE datasets is inspiring and proved to be effective.\n2. The experimental results are promising, and have the potential to be adopted in other tasks requiring tedious annotation schemas.\n3. Extensive demonstrations have been shown to prove the effect of the generator trained over different criteria.", "reasons_to_reject": "It would be even more interesting to see how this method could benefit other structured prediction tasks such as information extraction.\n", "questions_to_authors": "1. The alignment discriminator is trained by composing aligned and disaligned examples based on beam search and confidence scores. I wonder what is the beam size and the threshold confidence scores for selecting the negative samples. And how do you decide on these values? \n2. Fig 5 shows the trend by changing number of augmentations. What about changing the confidence thresholds?\n3. Fig 4 indicates the trends with both scores. Despite the scores given by two models, how about the actual fluency and alignment scores? Even a small set with human-evaluated scores on fluency and alignment is beneficial to see whether the discriminators actually align with humans' perceptions.", "paper_topic_and_main_contributions": "This paper proposes a novel way of augmenting data based on labels and syntactic templates to enhance the task of aspect sentiment triplet extraction. The augmentation is achieved by a generator which takes labels and a dependency template as input, and trained via reinforcement learning based on two other discriminators spanning over fluency and alignment scores. \n\nExtensive experiments have been conducted to demonstrate consistent performance gains across different datasets and extraction methods. And additional insights have been provided that showcased the influence of the fluency and alignment scores on the final performance.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "cQNsandiGh", "reviewer": "Reviewer_NQZC", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our great appreciation for your valuable comments to improve the quality of this manuscript.\n\n**Weakness-1: Limited Benchmark Datasets.**\n\nResponse-1: Thank you for your valuable feedback. We acknowledge including experimental results on 15Rest and 16Rest can further demonstrate the effectiveness of our approach. We are grateful for your suggestion and will refine this manuscript by incorporating the results of these two datasets.\n\n**Weakness-2: Baseline Selection.**\n\nResponse-2: We sincerely appreciate your feedback and thoughtful suggestions. As there are currently no data augmentation methods specifically designed for the ASTE task, we replicate three typical methods from other tasks for comparison. These three methods correspond to traditional rule-based methods, label-unrelated modification methods, and conditional language modeling methods. We appreciate your kind reminder on this matter and apologize for missing out on other methods. In response to this, we will refine this manuscript by providing an explanation for our rationale behind selecting these baselines and including additional baselines, such as DAGA.\n\nFinally, we want to express my sincere gratitude for your insightful and constructive feedback once again. Your input has played a crucial role in improving our work, and we deeply value your thoughtful involvement.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper is clearly written and easy to follow.\n2. The experiments involves several representative ASTE models and compare with several popular data augmentation baselines, to show the efficacy of the method.\n3. The idea is simple and effective.", "reasons_to_reject": "1. Limited benchmark datasets. This paper only performs experiments on two ASTE benchmarks. Maybe adding 15Rest and 16Rest of ASTE-Data-v2 would be better to show the approach is effective.\n2. Baseline selection. It is unclear how the authors choose the data augmentation baselines. Since ASTE includes aspect term extraction and opinion term extraction which are sequence labelling tasks. Readers might be curious whether authors compare the proposed method with some general sequence labelling-related data augmentation such as DAGA (Ding et al.,).\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a novel data augmentation method for Aspect Sentiment Triplet Extraction (ASTE), where they focus on learning a generator that can directly generate new sentences based on labels and syntactic templates.  To ensure the quality of the generated sentence, they introduce fluency and alignment discriminators to provide feedback on the generated sentence and then use the feedback to optimize the generator via a reinforcement learning framework. Extensive experiments show this approach improves the performance of several ASTE models.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Authors proposed an approach focusing on learning a generator that can directly generate new sentences by mixing labels and syntactic templates from different samples. The soundness and excitement are both 2 strong (4) and 1 good (3). Authors have responded and answered the questions raised by reviewers. This paper trains to generate samples for dataset augmentation and this method is shown to be effective. It can be helpful to the community and could be applied to other tasks.
Decision: Accept-Main