Smart “Chef”: Verifying the Effect of Role-based Paraphrasing for Aspect Term Extraction
Authors: Jiaxiang Chen
Yu Hong
Qingting Xu
Jianmin YAO
Venue: EMNLP 2023 Findings (2023)
Keywords: Information Extraction
Aspect Term Extraction
ChatGPT

Abstract: We tackle Aspect Term Extraction (ATE), a task of automatically extracting aspect terms from sentences. The current Pretrained Language Model (PLM) based extractors have achieved significant improvements. They primarily benefit from context-aware encoding. However, a considerable number of sentences in ATE corpora contain uninformative or low-quality contexts. Such sentences frequently act as "troublemakers" during test. In this study, we explore the context-oriented quality improvement method. Specifically, we propose to automatically rewrite the sentences from the perspectives of virtual experts with different roles, such as a "chef" in the restaurant domain. On this basis, we perform ATE over the paraphrased sentences during test, using the well-trained extractors without any change. In the experiments, we leverage ChatGPT to determine virtual experts in the considered domains, and induce ChatGPT to generate paraphrases conditioned on the roles of virtual experts. We experiment on the benchmark SemEval datasets, including Laptop-domain L14 and Restaurant-domain R14-16. The experimental results show that our approach effectively recalls the inconspicuous aspect terms like "al di la", although it reduces the precision. In addition, it is proven that our approach can be substantially improved by redundancy elimination and multi-role voting. More importantly, our approach can be used to expand the predictions obtained on the original sentences. This yields state-of-the-art performance (i.e., F1-scores of 86.2%, 89.3%, 77.7%, 82.7% on L14 and R14-16) without retraining or fine-tuning the baseline extractors.

Smart "Chef": Verifying the Effect of Role-based Paraphrasing for Aspect Term Extraction
Jiaxiang Chen, Yu Hong', Qingting Xu, Jianmin Yao School of Computer Science and Technology, Soochow University, SuZhou, China {xxfz56, tianxianer, qtxu0801 }egmail com, jyao@suda. edu. com com
Abstract
domain-specific attribute. For example, the text span "al di la" in (1) is specified restaurant- domain aspect term because it appears as the sign of an Italian trattoria.
We tackle Aspect Term Extraction (ATE), task of automatically extracting aspect terms from sentences. The current Pretrained Language Model (PLM) based extractors have achieved significant improvements. They primarily ben- efit from context-aware encoding. However, considerable number of sentences in ATE cor- pora contain uninformative or low-quality con- texts. Such sentences frequently act as "trou- blemakers" during test. In this study, we plore the cortext-oriented quality improvement method. Specifically, we propose to automat- ically rewrite the sentences from the perspec- tives of virtual experts with different roles, such "chef" in the restaurant domain. On this basis, we perform ATE over the paraphrased sentences during test, using the well-trained extractors without any change. In the experi- ments, we leverage ChatGPT to determine vir- tual experts in the considered domains, and induce ChatGPT to generate paraphrases con- ditioned on the roles of virtual experts. We ex- periment the benchmark SemEval datasets, including Laptop-domain L14 and Restaurant- domain R 14-16. The experimental results show that approach effectively recalls the incon- spicuous aspect terms like "al di la", although reduces the precision. In addition, is proven that approach can be substantially improved by redundancy elimination and multi-role vot- ing. More importantly, our approach can be used to expand the predictions obtained on the original sentences. This yields state-of-the art performance (i.e., I'1 -cc-sc of 86.2%, 89.3%, 77.7%, 82.7% on L14 and R14-16) without re- training or fine-tuning the baseline extractors.
(1) Love al di la (Selected from SemEval-R15),
(2) We take pride in every dish we serve al di la (Rewritten by ChatGPT with a role of "chef").
The current studies leverage PLMs as backbones to construct ATE models (extractors), including BERT (Devlin al., 2019), BART (Lewis et al., 2020) and T5 (Raffel al., 2020) mentioned in Section 2. They yield significant improvements, compared to conventional neural networks like CNN (LeCun et al., 1998). The advantage pri- marily attributed to the strong perception ability on noteworthy contexts, as well as context-aware representation learning ability.
However, such extractors frequently suffer from uninformative or low-quality contexts. For exam- ple, the context "Love" in (1) uninformative for recognizing "al di la". By contrast, the substitu- tion containing a knowledge-richt context makes it easier to recognize aspect terms, such as the case in (2). Accordingly, we propose ChatGPT-based Edition Fictionalization (CHEF) method to assist the current PLM- extractors. CHEF acts domain-specific virtual expert with different roles to rewrite sentences, with the aim to refine contexts of potential aspect terms. ChatGPT is utilized for both expert generation and sentence rewriting. A series of post-processing methods are coupled with CHEF including redundancy elimination, synonym replacement and multi-role voting.
1 Introduction
We experiment the benchmark datasets L14 and RI 14-16 (Pontiki al., 2014, 2015, 2016). The well-trained BERT base and PST (Wang et al., 2021) (SoTA) are adopted in the experiments. During test, they perform over the rewritten sentences by CHEF, without retraining and fine-tuning. The test results
ATE is a natural language processing task, which aims to extract aspect terms from sentences (Jakob and Gurevych, 2010). The aspect term refers to word, phrase or named entity depicting certain
"Corresponding author.

---

demonstrate the effectiveness of CHEF in recalling aspects and expanding the predictions.
Domain
Virtual Experts
Restaurant Owner, Chef. Waiter, Diner,
Catering Consultant, Receptionist.
Restaurant
Cleaning Staff, Purchasing Manager,
2 Related Work
Restaurant Manager, Financial Officer
Hardware Engineer, Software Engineer, Technical Support Engineer. Marketer, Customer Service Staff, Test Engincer
Context-aware encoding contributes to ATE. brings domain-specific contextual information into token-level representations. Therefore, CNN (Le- Cun et al., 1998; Karimi et al., 2021) and BERT (Devlin et al., 2019; Karimi et al., 2021; Klein et al., 2022) are widely used for ATE due to the abilities of convolving local contexts or absorbing attentive contextual information. Their expanded versions DECNN (Xu al., 2018; Wei al., 2020; et al., 2020; Chen and Qian, 2020) and BERT- PT (Xu et al., 2019; Wang et al., 2021; Chen et al., 2022b) are generally adopted as backbones in the subsequent studies. In addition, BERT is utilized the pedestal for context-aware encoding in series of more complex tasks, including Aspect- Sentiment Triplet Extraction (ASTE) (Chen et al., 2022a; Zhang al., 2022b; Chen al., 2022c; Zhang al., 2022a; Chen al., 2022d; Zhao et al., 2022b) and MRC-based ASTE (Yang and Zhao, 2022; Zhai et al., 2022). Recently, the generative framework is introduced into the studies of ASTE, and accordingly BART (Yan et al., 2021; Zhao et al., 2022a) and T5 (Zhang et al., 2021; Hu et al.. 2022) are used. They are constructed with the trans- former encoder-decoder architecture in the seq2seq paradigm, where attentive contextual information absorption is conducted on both sides.
Laptop
Case Designer, Supply Chain Manager, Purchasing Manager, User
Table I: Experts of Restaurant and Laptop domains.
Algorithm 1: Rewrite with Zero-shot prompting
Input:
Test set S, Role set R AND Domain set Output: All the rewritten sentences
Prompt.fimt(D,.ee. R..)) = Rewrite Sy E S from the perspective of R2 E R in the domain of D2 E D
foreach D; E 2D do
foreach R, E R do
foreach S S do
Instruction Prompt.fml Sk))
Prediction = ChatGPT(nssrnctioo))
end
end end
We consider two PLMs for hidden-state compu- tation in the experiments, including BERT base and BERTpt-base PST (Wang et al., 2021).
3.2 CHEF
CHEF comprises two stages, including rolc gen- eration of domain-specific virtual experts, as well as role-based rewriting. It is coupled with three post-processors, including redundancy elimination, synonym replacement and multi-role voting.
3 Approach
Role Generation- CHEF induces ChatGPT! to generate a series of virtual experts playing different roles. The generation is prompted by the target- domain name D, like "Restaurant". The query use is as follows: "Outpui the roles of experts in the domain of [D,] according to the different responsibiliiies." Table shows the experts.
We aim to provide knowledge-rich and high-quality sentences for ATE. Specifically, we use CHEF to rewrite sentences, and feed the rewritten sentences into an extractor to predict aspect terms. Finally, we combine the aspect terms which are respectively extracted from the original and rewritten sentences.
Sentence Rewriting Given sentence Sž in the ATE datasets, CHEF induces ChatGPT to rewrite the sentence from the perspective of a role-specific expert R,. Zero-shot prompting (Wei et al., 2022) is used during generation. In other words, there isn't t any example provided for prompting Chat- GPT. The query use is follows: "Rewrite the sentence [ the perspective of [R,] the domain of [D We rewrite all the instances in the test sets using Algorithm
3.1 Extractors
We follow the common practice (Chemyysevvich, 2014; Toh and Wang, 2014; San Vicente et al., 2015) to treat ATE as a sequence labeling task. B/T/O labels are used, which respectively signal Beginning, Inside and Outside tokens relative to aspect terms. Therefore, extractor essentially classifies each token into one of the B/1/O labels in terms of the token-level hidden state. We use PLM to compute the hidden states of tokens, and use a Fully-connected (FC) layer for classification.
Post-Processing- We drive the extractors to pre-
'htts://platfommopomoee wpwpaa/...mmllyyyyoommmmdddddaaa

---

1,-14
R-14
R-15
R-16
Train
Dov
Test
Train
Dev
Test
Train
Dev
Test
Train
Dev
Test
#Sentence #Aspect
2,436
609
800
2,433
608
800
1,052
263
685
1,600
400
676
1,940
418
654
2,950
744
1,134
948
252
542
1,408
336
612
Table 2: Statistics of ATE datasets. #Sentence and #Aspect denote the number of sentences and aspects.
dict aspect terms over the role-specific rewritten sentences, Redundant results may obtained, which are specified as the aspect terms never occurring in the original sentences. For example, although the token "dish" in the rewritten sentence in (2) is cor- aspect term, it is redundant due to non-occurrence in the original sentence in (1). We filter the redundant results during test.
sentences of all experts: V,=Ne/Nall. where No denotes the number of voting for acceptance, while
Nall is the number of experts. Nall is set to 10 in our experiments. For the Restaurant domain, a controversial result is finally adopted only if is less than threshold of 0.7. For the Laptop domain, the threshold is set to 0.8.
rectly predicted
Experimentation
In addition, we use a soft synonym replacement method to reduce false positive rates. Specifically, given an original sentence S; and the rewritten case Sü, we segment both of them into n-grams. As- sume set Ui of ne-ramms (1 < < 5) in S; share part of the predicted aspect term with the set U, of n-grams in S,, we calculate the similarity between each gram Mij in Mi and every gram üisk in 24, We rank all pairs of [Uij.ūgs] J in terms of their similar- ities, and select the top-1 ranked n-gram pair for synonym replacement, i... üز < Wij. Meanwhile, the replaced n gram is specified as the unabridged aspect term, shown in the example in (3). Note that Cosine similarity is computed over the em- beddings of each pair (ug,ūux). ). The embedding of each n-gram obtained by conducting mean pooling (Reimers and Gurevych, 2019) over the token-level hidden states.
4.1 Datasets and Evaluation
We experiment the SemEval datasets, including L14 and R14-16 (Pontiki al., 2014, 2015, 2016). All the instances L14 are selected from the Lap- top domain, while those in R14-16 derive from the Restaurant domain. We follow the common prac- tice (Dai and Song, 2019; Wang et al.. 2021) to split the datasets into training, validation and test sets. The statistics in them are shown in Table 2. It is noteworthy that only the extractors are trained and developed using the above datasets. CHEF has nothing to do with training and devel- opment. It conducted only on the test sets, pro- viding rewritten sentences for the extractors and post-processes the predictions. We evaluate all the models using Fl-score (Chernyshevich, 2014).
4.2 Hyperparameters
(3) Original: Best Indian food have ever eaten. Rewritten: I put a lot of effort into perfecting our Indian dishes [Aspect Term]. (Rewritten by CHEF with role of "chef")
We respectively use BERT base and BERT bbaseed PST (Wang et al., 2021) to construct the extractors. where PST achieved the best performance far. For BERT BoRTb we set the maximum sequence length to 128 and carry out training in 4 epochs. The optimization of model parameters obtained using AdamW with a learning rate of 3e-5 and a weight decay of 0.01. We set the batch size to 10. For PST, we adopt their initial hyperparameters, setting the first round of training to 5 epochs and performing 4 rounds of self- telf-training. The learning rate is set to 5e-5, AdamW used the optimizer. All the other hyperparameters remain consistent with the reported ones.
Replacement: Indian dishes < Indian food Output: Indian food
Multi-role Voting- We conduct multi-role vot- ing only an extractor obtains controversial results from the original and rewritten sentences. facili- tates the combination of the extraction results.
Assume an extractor refuses to extract text span f an aspect from the original sentence, though it would like to do from the sentences rewrit- ten by different roles of experts, thus we regard I; a controversial result. In this case, we define the behavior of extracting 1 as the voting for ac- ceptance, otherwise rejection. On this basis, we compute the acceptance rate in over the rewritten
4.3 Comparison Result
In Table 3, we report the performance of PST which is enhanced by CHEF, along with other ot

---

Model
1.14
R14
R15
R16
Baseline
+5-shot ICI. (2023)
ChatGPT (2023)
+5-shot COT (2023)
48.19
43.03
54.50
72.41
70.99
55.65
59.27
40,33
53.49
Recall rate (%)
Shrketer
Increuse
rate (%)
60
Hardnar Eagent
Panckalte
Increase
Parchnits
Maage
DECNN (2018)
81.59
74.37
40
+ CDA (2020)
81.58
75.19
+ Repositioning (2020)
84.17
84.55
72.03
75.40
L14
R14
R15
R16
L14
R14
R15
R16
ProcotypeF: (2020)
83.19
87.39
73.27
76.98
(a) Best Expert
(b) Worst Expert
BFRT base (2019)
79.86
86.58
68.08
73.50
+ +Repositioning (2020)
81,43
87.10
72.68
77.71
Figure 1: Significant improvements of Recall rates
CDA (2020)
81.14
75.89
yielded by CHEF the incompletely-solee sentences.
+PST (2021)
84.17
87.63
72.81
77.09
p (2019)
84,37
88.41
73.66
78.29
+CDA (2020)
85.33
80.29
+PST (2021)
86,91
88..75
75.82
82.56
Ours (PST+CHEF)
86.22
89.33
77.69
82.74
Table 3: The performance (F1--core) of various meth-
Fl-curee (56)
Fl-scure (%6
ods. "ICL" refers in-context learning, while "COT" stands for chain-of-though.. The sign "4" represents certain method combined with the baseline model.
-LI-BERT
-RI-BERT
SLIS
Model
L14
R14
R15
R16
-L1--CHE
-R1-CHHF
BERT 82.96
88.03
72.17
74.88
+ +CHEF 83,39
88.25
72.92
75,56
PST (Reproduced)
86.13
89.28
77.29
82.62
CHEF
86.22
89.33
77.69
82.74
Table 4: Improvements (F1) on different extractors.
58
23
Ti-soore (%6
Fl-goue (16)
71.32
17.51
the-art ATE models. In this case, the extraction results obtained by combining the predictions of PST both original and rewritten sentences, where multi-role voting is used.
be found that
RIS-BERT
-RIG-FRRT
CHEF enables PST to achieve better performance, slightly increasing the performance gap relative to other strong competitors.
RIS-CHEF
RI6-CHEF
54.55
In Table 4, we report the effects of CHEF on both the BERT base and BERT based PST, where multi- role voting is used for prediction combination. It be observed that CHEF enables both extractors achieve better performance on all the test sets, without retraining and fine-tuning.
Figure 2: F1-scores for the sentences at different lengths. Appendix A gives the statistics about them.
shows the experimental results, where only the performances yielded by the best and worst experts are provided. It can be observed that CHEF sub- stantially improves the recall rates for test sets, no matter whether it plays the role of best expert or worst. The most significant improvement in recall rate reaches about 9%. Besides, the change of pre- cision, recall and F1 score on the full test set can be found in Appendix B.
4.4 Discussion
In two separate experiments, we demonstrate that CHEF contributes the salvage of the missed as- pect terms, and yields more substantial improve- ments on short or long sentences.
Salvage Rate- We select all the incompletely- solved sentences from the test sets, each of which contains least one aspect term neglected by the extractor. On this basis, we use CHEF to rewrite the sentences and drive the extractor to rework them. In this experiment, we consider the BERT buse extrac- tor, and verify the changes in recall rates. Figure
Adaptability- CHEF applies more short and long sentences. The former generally contains unin- formative contexts, while the latter contains lower- quality contexts due to noises. We split each test set into different subsets according to the lengths sentences. There are five subsets obtained for each,

---

5 Conclusion
We utilize ChatGPT to rewrite sentences with dif- ferent roles of domain- sppciiic experts, provide informative and high-quality contexts for PLM-based ATE models. Experiments show that the proposed method contributes to the salvage of the neglected aspect terms, and applies more to the short and long sentences. In the future, we will use the rewritten sentences for contrastive learning. To reduce the reliance on ChatGPT, we will develop an offline context rewriting method by knowledge distillation and domain-specific pretraining.
Fi (%6)
25.06
12.00
Flacore
$1.55
$7.24
4
8033
Comb
Comb RE
Comb RE +R
ALL
Comb
Coon RE
Comb RE SR
ALL
Best Expert
(b) Worst Expert
Limitations
Figure 3: Results of ablation experiments (/l (Sscocces)).
We propose to use ChatGPT as an auxiliary toolkit to produce informative and high-quality contexts for context-aware token-level encoding, enhance PLM encoders for aspect term recogni- tion. Our experiments show that the proposed method yields slight improvements when coupled with strong Nomain-speciff models, and not only effective in recalling neglected cases, but per- forms better for short and long instances. Unavoid- ably, the proposed method has the limitations in building a self-contained model, due to the lack of training and fine-tuning. To overcome the problem, will develop lite comparable generator Chat- GPT by knowledge distillation and domain-specinn pretraining. Furthermore, we will incorporate role- based rewritten sentences into the training process. with the support of contrastive learning.
including the sentences having length within the ranges of [1, 10), [11, 15], [16, 20), [21, 30) and no less than 30. The statistics in the subsets shown in Table 5. Figure 2 shows the ATE performance at the original sentences of different lengths, and that at the rewritten cases by CHEF (without multi- role prediction combination). can be observed that CHEF only yields improvements for relatively short or long sentences. The most significant im- provement reaches about 4% Fl-score R16.
4.5 Ablation Study
CHEF consists of four components, including Combination (Comb), Redundancy Elimination (RE). Synonym Replacement (SR) and Multi-role Voting, as presented Section To verify the effects of the components, we conduct an ablation experiment. Figure 3 illustrates the verification results over best and worst experts, where ALL indicates the complete CHEF method.
Acknowledgment
The research is supported by National Key R&D Program of China (2020YFB1313601 and Na- tional Science Foundation of China (62376182, 62076174).
It can be observed that the simple combination (Comb) causes significant performance degrada- tion, although many terms the baseline missed are salvaged. When redundancy elimination (RE) is used, the comparable (a little worse) performance to the baseline is achieved. At this time, the preci- sion is still lower because the synonymous terms are regarded as negative examples during evalua- tion. When synonym replacement (SR) is used, the performance is increased for some roles while not for others. When multi-role voting used, the dis- agreement among the roles resolved, and thus the performance increased to a relatively higher level. We provide examples in Appendix C to facilitate the understanding of our ablation study.
References
Hao Chen, Zepeng Zhai, Fangxiang Feng, Ruifan Li, and Xiaojie Wang 2022a. Enhanced multi-channel graph convolutional network for aspect sentiment triplet extraction. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics, pages 2974-2985, Dublin, Ireland. Asso- ciation for Computational Linguistics.
Jiaxiang Chen, Yu Hong, Qingting Xu, Jianmin Yao, and Guodong Zhou. 2022b. Enhancing neural aspect syntactic
term extraction using pendency features. In 2022 IEEE 34th International

---

Conference on Tools with Artificial Intelligence (IC- TAI), pages 503-310.
2010 Conference on Empirical Methods Natural Language Processing, pages 1035-1045, Cambridge. MA. Association for Computational Linguistics.
Wei Chen, Jinglong Du Zhao Zhang, Fuzhen Zhuang. and Zhongshi He. 2022c. A hierarchical interactive network for joint span-based aspect-sentiment anal- ysis. In P Proceedings of the 29th International Con- ference on Compulational Linguistics, pages 7013- 7019, Gyeongju. Republic of Korea. International Committee on Computational Linguistics.
Akbar Karimi, Leonardo Rossi, and Andrea Prati. 2021. Adversarial training for aspect-based sentiment anal- ysis with bert. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 8797-8803.
Ayal Klein, Oren Pereg, Daniel Korat, Vasudev Lal, Moshe Wasserblat, and Ido Dagan. 2022. Opinion- based relational pivoting for cross-domain aspect term extraction. In Proceedings of the 12th Work- shop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, pages 104-112, Dublin, Ireland. Association for Computational Lin- guistics.
Yuqi Chen, Chen Keming, Xian Sun, and Zequn Zhang.
2022d. A span-level bidirectional network for as- pect sentiment triplet extraction. In Proceedings of the 2022 Conference Empirical Methods in Nat- urul Language Processing, pages 4300-4309, Abu Dhabi, United Arah Emirates. Association for Com- putational Linguistics.
Zhuang Chen and Tieyun Qian. 2020. Enhancing aspect term extraction with soft prototypes. In Proceedings of the 2020 Conference on Empirical Methods in Nat- ural Language Processing, pages 2107-2117, Online. Association for Computational Linguistics.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-basee learning applied to document recognition. Proceedings of the IEEE, (6(11))2278-232)
Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising grquence-to-squenee pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguisties. pages 7871-7880) Online. Association for Computa- tional L Linguistics.
Maryna Chernyshevich.. 2014. IHS R&D Belarus: Cross-domain extraction of product features using CRF. In Proceedings of the 8th International Work- shop on Semantic Evaluation (SemEval 2014), pages 309-313, Dublin, Ireland. Association for Computa- tional Linguistics.
Hongliang Dai and Yangqiu Song. 2019, Neural aspect and opinion term extraction with mined rules weak supervision. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 5268-5277, Florence, Italy. Association for Computational Linguistics.
Kun Li, Chengbo Chen, Xiaojun Quan, Qing Ling, and Yan Song. 2020. Conditional augmentation for aspect term extraction via masked sequence-to- to- sequence generation. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 7056-7066, Online. Association for Computational Linguistics.
Jacob Devlin, Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies. pages 4171-4188 Minneapolis, Minnesota. Association for Computational Linguistics.
Maria Pontiki, Dimitris Galanis, Haris Papagcorgiou, Ion Androutsopoulos, Suresh Manandhar, Moham- mad AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clereq. Véronique Hoste, Marianna Apidianaki. Xavier Tannier, Na- talia Loukachevitch, Evgeniy Kotelnikov, Nuria Bel, Salud María Jiménez-Zafra, and Gülşen Eryigit. 2016. SemEval-2016 task 5: Aspect based senti- ment amalysis. In SemFvval 2016, pages 19-30, San Diego, California.
Ridong Han, Tao Peng, Chaohao Yang, Benyou Wang. Lu Liu, and Xiang Wan. 2023. Is information extrac- tion solved by chatgpt? an analysis of performance, evaluation criteria, robustness and errors. arXiv preprint 2ix: 14450.
Mengting Hu, Yike Wu, Hang Gao, Yinhao Bai, and Shiwan Zhao. 2022. Improving aspect sentiment quad prediction via template-order data augmenta- tion. In Proceedings of the 2022 Conference on Em- pirical Methods in Natural Language Processing, pages 7889-7900, Abu Dhabi, United Arah Emirates. Association for Computational Linguistics.
Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and lon Androutsopoulos. 2015. SemEval-2015 task 12: Aspect based sentiment anal- ysis. In SemEval 2015, pages 486-495, Denver, Col- orado.
Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Har- ris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In SemEval 2014, pages 27-35, Dublin, Ireland.
Niklas Jakob and Iryna Gurevych. 2010. Extracting opinion targets single and cross-domain setting with conditional random fields. In Proceedings of the

---

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res., 11:140:1-140:66
Hang Yan, Junqi Dai, Tuo Ji, Xipeng Qiu, and Zheng Zhang. 2021. A unified generative framework for aspect-based sentiment analysis. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1ith International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 2416-2429, Online. Association for Computational Linguistics.
Nils Reimers and Iryna Gurevych. 2019. Sentence- BERT: Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods Natural Language Processing and the 9th International Joint Conference on Natu- val Language Processing (EMNLP-IJCNLA) pages 3982 3992, Hong Kong. China, Association for Com- putational Linguistics.
Yifei Yang and Hai Zhao. 2022. Aspect-based senti- ment analysis as machine reading comprehension. In Proceedings of the 29th International Conference on Compitational Linguistics, pages 2461-2471. Gycongju, Republic of Korea. International Com- mittee on Computational Linguistics.
Iñaki San Vicente, Xabier Saralegi, and Rodrigo Agerri. 2015. EliXa: A modular and flexible ABSA plat- form. In Proceedings of the 9th International Work- shop on Semantic Evaluation (SemEval 2015), pages 748-752, Denver, Colorado. Association for Compu- tational Linguistics.
Zepeng Zhai, Hao Chen, Fangxiang Feng, Ruifan Li, and Xiaojie Wang. 2022. COM-MRC: A COntext- masked machine reading comprchension framework for aspect sentiment triplet extraction. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3230-3241, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Zhiqiang Toh and Wenting Wang. 2014. DLIREC: As- pect term extraction and term polarity classification system. In Proceedings the 8th International Work- shop on Semantic Evaluation (SemEvul 2014), pages 235-240, Dublin, Ireland. Association for Computa- tional Linguistics.
Chen Zhang. Lei Ren, Fang Ma, Jingang Wang, Wei Wu, and Dawei Song. 2022a. Structural bias for aspect sentiment triplet extraction. In Proceedings of the 29th International Conference Computational Linguistics, pages 6736-6745, Gyeongju, Republic of Korea. International Committee on Computational Linguisties.
Qianlong Wang, Zhiyuan Wen, Qin Zhao, Min Yang. and Ruifeng Xu. 2021. Progressive self-training with discriminator for aspect term extraction. In Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 257-268, Online and Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics.
Wenxuan Zhang, Xin Li, Yang Deng, Lidong Bing, and Wai Lam. 2021. Towards generative aspect-based sentiment analysis. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing, pages 504- 510, Online. Association for Computational Linguis- tics.
Jason Wei. Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V Le. 2022. Finetuned language mod- els zero-shot learners. In International Confer- ence on Learning Representations.
Yice Zhang, Yifan Yang. Yihui Li, Bin Liang, Shi- wei Chen, Yixue Dang, Min Yang, and Ruifeng Xu. 2022b. Boundary-driven table-filling for aspect sen- timent triplet extraction. In Proceedings of the 2022 Conference on Empirical Methods in Natural Lan- guage Processing, pages 6485-6498, Abu Dhabi, United Arab Emirates. Association for Computa- tional Linguistics.
Zhenkai Wei, Yu Hong, Bowei Zou, Meng Cheng, and Jianmin Yao. 2020. Don't It cclipse your arts due to small discrepancies: Boundary repositioning with a pointer network for aspect extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3678-3684, 3684, Online. Association for Computational Linguistics.
Hu Xu, Bing Liu, Lei Shu, and Philip Yu. 2019. BERT pest-training for review reading comprehension and aspect-based sentiment analysis. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Pupers), pages 2324-2335, Minneapolis, Min- nesota. Association for Computational Linguistics.
Shiman Zhao, Wei Chen, and Tengjiao Wang. 2022a. Learning cooperative interactions for multi-overlan aspect sentiment triplet extraction. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3337-3347, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Yichun Zhao, Kui Meng, Gongshen Liu, Jintao Du, and Huijia Zhu. 2022b. A multi-task dual-tree network for aspect sentiment triplet extraction. In Proceed- ings of the 29th International Conference on Com- putational Linguistics, pages 7065-7074, Gyeongju, Republic of Korea. International Committee Com- putational Linguistics.
Hu Xu, Bing Liu, Lei Shu, and Philip S. Yu. 2018. Dou- ble embeddings and CNN-based sequence labeling for aspect extraction. In Proceedings of the 56th An- nual Meeting of the Association for Computational Linguistics, pages 592-598, Melbourne, Australia. Association for Computational Linguistics.

---

[1. 10]
[11,
15
[16, 20]
[21. 30]
>> 30
#Sentence
#Aspect
#Sentence
#Aspect
#Sentence
#Aspect
#Sentence
#Aspect
#Sentence
#Aspect
L14
260
126
224
168
158
115
122
80
R14
218
190
238
287
151
241
137
253
163
R15
254
153
148
130
111
108
103
116
69
90
R16
240
161
149
138
125
130
120
101
Table 5: Statistics of ATE test sets split according to the lengths of sentences.
1.14
R14
R15
R16
C Examples for Ablation Study
85.97
88.23
70.28-770007
74.76-474.37
R 10.15-81..35
88.85
89.
74.17-70011
75.00-+76.80
Our method combines the extracted results from both the original and rewritten sentences. However, the combination without post-processing causes ter- rible performance because of noises. To solve the problem, have introduced the post-processigg method, including redundancy elimination, syn- onym replacement, and multi-role voting. We pro- vide examples that have been post-processed using the above solutions in Table
F1 82.88-2.33... 12..011--...35777777777.. 711-7722277.88775..2. -75.56
Table Performance changes of Precision, Recall and Fl Fl-score when applying our method.
Examples
Example of redundancy elimination:
Original: Better than nobu, tomoe, taka, yama, sugiyama, and the rest.
Waiter: chef's sushi so amazing, it's better than Nobu, Tomoc, Taka, Yama, Sugiyama, and other well-
known sushi spots in the city.
Ground-truth; (No Aspect]
Output from the rewritten sentence: chef'ss sushi
Example of synonym replacement:
Original: Best Indian food have ever caten.
Chef: put lot of effort into perfecting Indian dishes.
Ground-truthh Indian food
Output from the rewritten sentence: Indian dishes
Example multi-role voting:
Original: have to say they have one of the fastest delivery times in the city.
Chef: can prepare the order quickly ensure timely delivery.
Restaurant Owner: Our delivery times are some the fastest the city.
Catering Consultant: Their delivery times are impressive and contribute to positive customer experience. Ground-truth: delivery times
Output from the Chef: delivery
Table 7: Examples of how each component works.
Test Set Reorganization
We divide the test sets into different subsets accord- ing to the lengths of sentences. For each test set, we obtained five subsets, which can be found in Table 5.
Change of Precision and Recall
We provide additional results reflecting the change in precision and recall scores, shown in Table 6. It can be found that the precision score decreases. meanwhile, the recall score increases. The perfor- mance gain at the recall score is because of the enhancement from our post- processng methods.


Reviews:
[{"review_id": "EYcdOv5K9g", "reviewer": "Reviewer_p6WA", "summary": "", "questions": "1. What other groups of prompts were tried for this purpose?\n2. Does the model performance scale beyond these two domains?\n3. In the limitations section, there is a claim that \"the proposed method yields slight improvement when coupled with domain-specific models.\" Could you provide some qualitative examples to highlight this difference along with the domain-specific models in question?\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere gratitude for dedicating your valuable time to reviewing our paper and providing us with insightful questions and suggestions. We have meticulously considered each of your suggestions and comments. We answer the concerns as below:\n\n\n\n----------Response to Comments----------\n\n\n**Comment #1:** A more extensive comparison across models, including the current aspect extraction state-of-the-art as well as a detailed error analysis of this pipeline would have been useful.\n\n\n**Response to #1:** We conducted redundancy elimination, synonym replacement, and multi-role voting on the extracted results. These auxiliary techniques were implemented to ensure the overall effectiveness of our approach and the fairness of comparison with other methods.\n\nWe realize that the mentioned error analysis aligns with the ablation study, where the synonym replacement and multi-role voting need to be ablated progressively for insight into the performance in different phrases. We provide a detailed ablation study for every generated role in Table 1 in the Appendix behind our replies. Besides, we provide examples to assist the observation.\n\nDue to the page limit, we haven’t provided exhaustive contents. In the extended version, we will be delighted to present our more comprehensive experimental data.\n\n\n\n----------Response to Questions----------\n\n\n**Question #1:** What other groups of prompts were tried for this purpose?\n\n\n**Response to #1:** We didn't try other prompts. The sole prompt we tried at the very beginning works well. In other words, we consistently utilized a single prompt without employing techniques such as context learning, which made it easy to follow.\n\n\n**Question #2:** Does the model performance scale beyond these two domains?\n\n\n**Response to #2:** At present, our focus is restricted solely to the ABSA datasets from SemEval for Aspect Term Extraction (ATE) research. The datasets include only the restaurant and laptop domains.\n\nWe also realized other existing datasets that are relevant to ATE, such as the COQE task that involves the camera and car domains. However, we concentrate on pure ATE task research without any additional information interventions.\n\nWe are considering the application of this approach in future endeavors, specifically for tasks such as ABSA and COQE, in order to assess its reliability within an end-to-end system. During this phase, we can explore potential applications in other domains.\n\n\n**Question #3:** In the limitations section, there is a claim that \"the proposed method yields slight improvement when coupled with domain-specific models.\" Could you provide some qualitative examples to highlight this difference along with the domain-specific models in question?\n\n**Response #3:** We are so sorry that the section of limitations may have been misleading. We want to convey that our approach can yield performance improvements on a variety of \"domain-specific models\", where we mean those trained on a specific domain of the ATE task. In our experiments, we employed three models: BERT-base, PST (both detailed in the paper), and one that we are currently working on. Our approach demonstrated consistent improvements across all these models. We are enthusiastic about sharing the data in the final revision.\n\n\n\n----------Appendix----------\n\n\n**Table 1. Ablation Studies for Different Roles**\n| **Res14**                    | Restaurant Owner  | Chef              | Waiter        | Restaurant Manager | Purchasing Manager         | Catering Consultant  | Receptionist | Financial Officer  | Cleaning Staff | Diner                  |\n| ---------------------------- | ----------------- | ----------------- | ------------- | ------------------ | -------------------------- | -------------------- | ------------ | ------------------ | -------------- | ---------------------- |\n| BERT-base                    | 88.03             | 88.03             | 88.03         | 88.03              | 88.03                      | 88.03                | 88.03        | 88.03              | 88.03          | 88.03                  |\n| only combine (lines 117-118) | 70.65             | 61.47             | 68.82         | 65.51              | 51.94                      | 60.87                | 70.14        | 61.06              | 64.66          | 68.11                  |\n| redundancy elimination       | 87.10             | 87.67             | 87.28         | 87.15              | 86.60                      | 87.19                | 87.60        | 87.27              | 87.35          | 87.40                  |\n| synonym replacement          | 87.04             | 87.45             | 87.69         | 87.26              | 87.54                      | 87.22                | 87.80        | 87.69              | 87.27          | 87.25                  |\n| multi-role voting            | 88.25             | 88.25             | 88.25         | 88.25              | 88.25                      | 88.25                | 88.25        | 88.25              | 88.25          | 88.25                  |\n| **Res15**                    | **Restaurant Owner**  | **Chef**              | **Waiter**       | **Restaurant Manager** | **Purchasing Manager**         | **Catering Consultant**  | **Receptionist** | **Financial Officer**  | **Cleaning Staff** | **Diner**                  |\n| BERT-base                    | 72.17             | 72.17             | 72.17         | 72.17              | 72.17                      | 72.17                | 72.17        | 72.17              | 72.17          | 72.17                  |\n| only combine (lines 117-118) | 57.24             | 47.92             | 55.64         | 53.48              | 47.7                       | 49.65                | 56.68        | 53.72              | 45.66          | 50.68                  |\n| redundancy elimination       | 71.93             | 70.59             | 71.70         | 72.18              | 70.80                      | 71.67                | 71.64        | 71.55              | 71.01          | 71.56                  |\n| synonym replacement          | 71.93             | 71.00             | 71.54         | 71.31              | 70.85                      | 71.53                | 71.51        | 70.78              | 70.72          | 70.83                  |\n| multi-role voting            | 72.92             | 72.92             | 72.92         | 72.92              | 72.92                      | 72.92                | 72.92        | 72.92              | 72.92          | 72.92                  |\n| **Res16**                    | **Restaurant Owner**  | **Chef**              | **Waiter**       | **Restaurant Manager** | **Purchasing Manager**         | **Catering Consultant**  | **Receptionist** | **Financial Officer**  | **Cleaning Staff** | **Diner**                  |\n| BERT-base                    | 74.88             | 74.88             | 74.88         | 74.88              | 74.88                      | 74.88                | 74.88        | 74.88              | 74.88          | 74.88                  |\n| only combine (lines 117-118) | 61.55             | 54.41             | 62.66         | 58.75              | 51.12                      | 55.58                | 62.56        | 58.09              | 56.03          | 57.14                  |\n| redundancy elimination       | 73.60             | 74.32             | 74.20         | 74.26              | 73.98                      | 73.90                | 73.72        | 73.50              | 73.98          | 74.51                  |\n| synonym replacement          | 74.87             | 74.04             | 74.48         | 74.54              | 74.53                      | 73.81                | 73.47        | 72.89              | 74.06          | 74.10                  |\n| multi-role voting            | 75.56             | 75.56             | 75.56         | 75.56              | 75.56                      | 75.56                | 75.56        | 75.56              | 75.56          | 75.56                  |\n| **Lap14**                    | **Hardware Engineer** | **Software Engineer** | **Case Designer** | **Test Engineer**      | **Technical Support Engineer** | **Supply Chain Manager** | **Marketer**     | **Purchasing Manager** | **User**           | **Customer Service Staff** |\n| BERT-base                    | 82.96             | 82.96             | 82.96         | 82.96              | 82.96                      | 82.96                | 82.96        | 82.96              | 82.96          | 82.96                  |\n| only combine (lines 117-118) | 42.99             | 42.17             | 40.93         | 51.41              | 55.03                      | 45.54                | 50.33        | 43.65              | 62.26          | 58.34                  |\n| redundancy elimination       | 82.16             | 82.32             | 81.89         | 82.59              | 81.71                      | 82.25                | 82.88        | 82.33              | 82.32          | 81.93                  |\n| synonym replacement          | 82.45             | 82.13             | 82.22         | 82.82              | 82.18                      | 82.53                | 82.84        | 82.83              | 82.81          | 82.04                  |\n| multi-role voting            | 83.39             | 83.39             | 83.39         | 83.39              | 83.39                      | 83.39                | 83.39        | 83.39              | 83.39          | 83.39                  |\n\n\n\n**Examples**\n\n**1. Example of redundancy elimination:**\n\n**Original**: Better than nobu, tomoe, taka, yama, sugiyama, and the rest.\n\n**Waiter**: Our **chef's sushi** is so amazing, it's better than Nobu, Tomoe, Taka, Yama, Sugiyama, and other well-known sushi spots in the city.\n\n**Ground-truth**: [No Aspect]\n\n**Output from the rewritten sentence**: chef's sushi\n\n\n\n**2. Example of synonym replacement**\n\n**Original**: Best **Indian food** I have ever eaten.\n\n**Chef**: I put a lot of effort into perfecting our **Indian dishes**.\n\n**Ground-truth**: Indian food\n\n**Output from the rewritten sentence**: Indian dishes\n\n\n\n**3. Example of multi-role voting**\n\n**Original**: I have to say they have one of the fastest **delivery times** in the city.\n\n**Chef**: I can prepare the order quickly to ensure timely **delivery**.\n\n**Ground-truth**: delivery times\n\n**Output from the rewritten sentence**: delivery\n\n\n\nWe are grateful for your insightful comments, which undoubtedly strengthen the quality of our paper. Your guidance has played a crucial role in refining our work, making it more coherent and impactful.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This is a novel method of using ChatGPT prompt engineering for a classification task\n2. The method seeks to be more controllable than traditional LLM-based classification methods, as the prompts used by ChatGPT are human-readable and editable, and the post-processing looks traditional and rule-based (including synonym replacement and multi-role voting for contentious results)\n3. The performance of the models shows promise that controlled prompting of ChatGPT-like models can help improve model performance across select classification tasks", "reasons_to_reject": "1. A more extensive comparison across models, including the current aspect extraction state-of-the-art as well as a detailed error analysis of this pipeline would have been useful.", "questions_to_authors": "1. What other groups of prompts were tried for this purpose?\n2. Does the model performance scale beyond these two domains?\n3. In the limitations section, there is a claim that \"the proposed method yields slight improvement when coupled with domain-specific models.\" Could you provide some qualitative examples to highlight this difference along with the domain-specific models in question?\n", "paper_topic_and_main_contributions": "This paper aims to improve aspect term extraction by performing automatic rewriting tasks for domain-specific aspect extraction using the lens of subject experts using ChatGPT. The methodology is tested on the SemEval task corpora L14 and R14-16 and shows an increase in F1-score and performance. The method, known as ChatGPT-based Edition Fictionalization, does not pretrain or fine-tune ChatGPT in this rewriting exercise with a list of virtual experts which are generated by ChatGPT using prompts, following which the method uses the generated list of roles as keys to rewrite the input sentence for classification.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hYuEablE95", "reviewer": "Reviewer_m3uM", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our heartfelt appreciation for dedicating your valuable time to reviewing our paper and offering us invaluable insights and suggestions aimed at enhancing its quality. Your questions have made us aware of the key to further improving the paper We have taken careful consideration of each of your suggestions and comments, and we hereby provide our responses to each point as below.\n\n\n\n----------Response to Comments----------\n\n\n**Comment #1:** The paper claims that CHEF can paraphrase the original sentence into a more informative and high-quality one, but during the process it also injects model bias from Chat-GPT into the generated sentence, which will shift the distribution of training data away from test data. This kind of noises can counteract the benefits from more wonderful data, and as it shows in Table 3 the improvements is not very significant probably due to this.\n\n\n**Response to #1:** We totally recognize your point of view. To address noises, we have introduced post-processing operations, including redundancy elimination, synonym replacement, and multi-role voting.\n\nOur method exclusively combines the extracted results from both the original and rewritten sentences and performs post-processing. It refrains from retraining the ATE model, thus avoiding any potential distribution shifting away from the test data. All our post-processing operations are straightforward to implement without retraining. \n\n\n**Comment #2:** The paper lacks an analysis or statement about the time/financial cost of the methods (e.g., the cost of Chat-GPT inference in terms of USD).\n\n\n**Response to #2:** Our method obviates the necessity for model retraining, requiring solely the rewriting of sentences within the test set, encompassing a total of 2,963 sentences. The request of ChatGPT for generating ten rewritten sentences for each test sample yielded an average response time of 2.5 seconds, resulting in an approximate total cost of $90. In forthcoming research, we plan to leverage ChatGPT to distill a LLaMA model for rewriting sentences, thereby mitigating API expenses.\n\n\n\nWe extend our heartfelt gratitude for your insightful comments, which will help us improve the quality of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The motivation of the paper is clear, sensible and evident to the proposed method.\n2. The proposed idea is overall novel and interesting, which effectively emploits an LLM to find experts and generate more high-quality data to facilitate the extractor.\n3. The writing is generally good, which I can easily follow. The examples displayed in tables or raw text makes the illustration very clear and easy to understand.", "reasons_to_reject": "1. The paper claims that CHEF can paraphrase the original sentence into a more informative and high-quality one, but during the process it also injects model bias from Chat-GPT into the generated sentence, which will shift the distribution of training data away from test data.  This kind of noises can counteract the benefits from more wonderful data, and as it shows in Table 3 the improvements is not very significant probably due to this.\n\n2. The paper lacks an analysis or statement about the time/financial cost of the methods (e.g., the cost of Chat-GPT inference in terms of USD).", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper focuses on the aspect triplet extraction task and proposes a ChatGPT-based Edition Fictionalization (CHEF) method to assist the current PLM-based extractors.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Wwye38h7EB", "reviewer": "Reviewer_1MTq", "summary": "", "questions": "A. How sensitive are the empirical results from perturbing the multi-role voting related hyperparameter values? Especially, the overall empirical results carry some level of \"still do no harm in worst case\" argument, and I am curious whether this is an empirically robust claim.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere appreciation for your dedicated effort in reviewing our papers and offering valuable suggestions to enhance their quality. Your feedback holds significant value for us, and we genuinely consider each suggestion you provide. We take your feedback seriously, and our response is as follows:\n\n\n\n----------Response to Comments----------\n\n\n**Comment #1:** Empirical validation covers a narrow set of domains, and the reported improvements in Table 3 and Figure 2 are mostly positive but can be seen as finely tuned results. In addition to the hyperparameters explicitly discussed in section 4.2, the voting section (lines 201-208) seems to contain more hyperparameters whose sensitivity to the final result is not presented.\n\n\n**Response to #1:** Our research is centered on the Aspect Term Extraction (ATE) task, specifically involving reviews as input without any additional features or sentiment analysis considerations. To facilitate a comprehensive comparison with prior ATE studies, our primary emphasis lies on the four SemEval datasets, which cover only these two domains.\n\n\nOur investigation revealed that the sentences rewritten by ChatGPT occasionally introduced noises. For example, the results extracted from the rewritten sentences may contain some aspect terms that did not exist in the original text, although such terms are appropriate according to our verification. Since our evaluation was grounded in the ground-truth of the original text, we found it necessary to implement filtering. These adjustments were made to ensure a fair and equitable comparison with the work of others.\n\nThere is only one hyperparameter that needs to be set, which is the threshold limiting the number of different roles of virtual experts (the ones generated by ChatGPT) for voting. We used ChatGPT to automatically generate 10 virtual experts who have different roles in a specific domain, and allow 7 experts to join the voting process. This is an empirical setting according to the performance of the experts on a much smaller subset of data sampled from the validation set. \n\n\n\n----------Response to Questions----------\n\n\n**Question #1:** How sensitive are the empirical results from perturbing the multi-role voting related hyperparameter values? Especially, the overall empirical results carry some level of \"still do no harm in worst case\" argument, and I am curious whether this is an empirically robust claim.\n\n\n**Response to #1:** You raise a very important issue, the acceptance rate (line 200) is very sensitive to the final result.\n\n\nWe employed ChatGPT to generate ten experts without imposing stringent constraints. It's worth noting that, in certain instances, this approach may introduce noise, potentially leading to adverse effects on sentences. For instance, the purchasing manager might not be capable of providing valuable information regarding food taste. Consequently, we have devised an integrated system that employs a voting mechanism to address these concerns.\n\n\nWe manually observed a relatively positive effect of vote rate at 0.7 and 0.8 based on a small subset of data sampled from the validation set. We appreciate that an in-depth investigation into the roles' reliability presents interesting challenges.\n\n\nWe sincerely appreciate the constructive critique provided by you, which has undeniably enriched the final version of the paper. Thank you for your time and consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "It is a readily-applicable technique that leverages ChatGPT, and it can be applied to a broad class of methods to solve ATE tasks. The results in Table 2 seem to be a solid incremental result over previously published techniques. Straightforward idea, easy-to-use choice of tool, tangible improvements and potentially broad applicability make this paper a reasonable candidate. ", "reasons_to_reject": "Empirical validation covers a narrow set of domains, and the reported improvements in Table 3 and Figure 2 are mostly positive but can be seen as finely tuned results. In addition to the hyperparameters explicitly discussed in section 4.2, the voting section (lines 201-208) seems to contain more hyperparameters whose sensitivity to the final result is not presented.\n", "questions_to_authors": "A. How sensitive are the empirical results from perturbing the multi-role voting related hyperparameter values? Especially, the overall empirical results carry some level of \"still do no harm in worst case\" argument, and I am curious whether this is an empirically robust claim.", "paper_topic_and_main_contributions": "This paper presents a way to utilize ChatGPT to help pre-existing PLM-based tools for Aspect Term Extraction (ATE), through relevant-role-prompted rewriting of input sentences. This may be seen as using domain-specific prompt-engineering to augment input sentences, implemented with ChatGPT and empirical performance evaluated on four groups of sentences covering two domains (Laptops and Restaurants) from SemEval datasets.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YbS8CVgAXo", "reviewer": "Reviewer_FoJG", "summary": "", "questions": "A) Is ATE useful for something other than ABSA?  If no, have you done experiments comparing with end-to-end systems (presumably generative, such as Zhang et al. 2021)?  This would obviously require integration with an ABSA system which takes the aspect terms as input.  \n\nB) Do you have any data that addresses the question which of the components of your approach actually help?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We wholeheartedly appreciate your dedicated time in reviewing our paper and offering valuable suggestions to enhance its quality. Your feedback holds immense value for us, and we approach it with utmost seriousness. Each suggestion you provided has been thoroughly considered, and we provide our responses as follows:\n\n\n\n----------Response to Comments----------\n\n\n**Comment #1:** The way of using ChatGPT is complex (and interesting), but the authors do not present an ablation study to show which steps help.\n\n\n**Response to #1:** We totally agree with your viewpoint. Owing to the page limit, we have not provided a comprehensive list of results. In the extended version, we are enthusiastic about sharing more extensive experimental data. In order to provide a quick insight for reviewers, we show the available data in Table 1 behind our replies, and provide examples (behind Table 1) to assist the analysis. Table 1 exhibits the performance of our approach when different experts are respectively used for paraphrase generation and result combination, as well as the effect of utilizing synonym replacement and redundancy elimination. \n\n\n**Comment #2:** The improvements are small, and in fact smaller than the change in results due to a re-implementation (if I understand correctly). While it is not the standard in our field to report statistical significance, when improvements are really small, it becomes perhaps more urgent to do so.\n\n\n**Response to #2:** Our efforts primarily focused on qualifying the input sentences at the inference stage. This resulted in a slight performance enhancement across various models. The improvements were obtained without changing the previously-trained ATE model, or retraining and fine-tuning it. It preliminarily proved our hypothesis that role-based sentence rewriting can impose positive effects on ATE. Based on the findings and experiments in our submission, there are most probably a variety of interesting and promising methods that can be soon proposed in the future, such as role-based data augmentation, role-based distracting data generation for generative adversarial mechanism, or attention remodeling during training.\n\nIn the Appendix behind our replies, we present the statistical significance analysis results, where P-value is used. We hope it can provide deeper insight into the significance. We intend to provide all the results in the extended revision of this paper.\n\n**Comment #3:** If, as this reviewer believes, aspect Term Extraction (ATE) is only useful as a pre-processing step for ABSA, then the reader would also like to see a comparison with end-to-end systems.\n\n\n**Response to #3:** We are actively implementing your recommendations. Specifically, we are in the process of employing ChatGPT to rewrite sentences for the partial training set. Using such data, we are training LLaMA by distillation. The goal is to produce a more efficient and cost-effective rewriter. Our next objective is to validate whether incorporating this data into the training process further improves performance. This will enable us to conduct more comprehensive experiments using end-to-end models.\n\n\n\n----------Response to Questions----------\n\n\n**Question #1:** Is ATE useful for something other than ABSA? If no, have you done experiments comparing with end-to-end systems (presumably generative, such as Zhang et al. 2021)? This would obviously require integration with an ABSA system which takes the aspect terms as input.\n\n\n**Response to #1:** We are conducting more complete experiments on the ATE task to more fully apply it to tasks such as ABSA and Comparative Opinion Quintuple Extraction (COQE).\n\n\n**Question #2:** Do you have any data that addresses the question which of the components of your approach actually help?\n\n\n**Response to #2:** We are enthusiastic about sharing these experimental data with you (Please see Table 1 and Examples in the Appendix at the end of the response).\n\n\n\n----------Response to Typos Grammar Style And Presentation Improvements----------\n\n\nWe greatly appreciate your valuable suggestions regarding our writing, and we are committed to incorporating them into the revised version of the paper to enhance its overall quality. \n\n\nBesides, we have meticulously considered each of your suggestions in the “Typos Grammar Style And Presentation Improvements”. We answer the concerns as below:\n\n\n**Suggestion #1:** Is this task performed only in the context of sentiment analysis, or is it also important for other tasks? Maybe make this clear from the beginning. (line 039)\n\n\n**Response to #1:** ATE is a subtask of ABSA, and the dataset employed in our study can analyze only ABSA or exclusively focus on ATE. We fully concur with your suggestion, if space permits, and intend to add the introduction at the beginning of the paper.\n\n\n**Suggestion #2:** Why? In the context of sentiment analysis, whatever follows “love” would be as aspect, no? (line 062)\n\n\n**Response to #2:** This is not an absolute situation. In the Res14 training set, we identified 46 instances of the term \"Love,\" where 27 occurrences of noun phrases following \"Love\" were not annotated as aspect terms. For example, \"I love the fact that the pizza tastes so good and is so cheap.\" In this example, the aspect term is \"pizza tastes,\" rather than \"fact\".\n\n\n**Suggestion #3:** I don’t understand this. Explain. What is “content”? (line 153)\n\n\n**Response to #3:** We were trying to explain that [S_k] is the content of the original sentence. Our prompt is “Rewrite the sentence [S_k] from the perspective of [R_j] in the domain of [D_i]”, where [S_k] will be replaced by a real sentence (i.e., original sentence) before the prompt is fed into ChatGPT.\n\n**Suggestion #4:** This is an old approach — why not use a generative approach? (line 120)\n\n\n**Response to #4:** We observed that the current studies of generative methods exclusively present ATE performance on the ASTE dataset, which is a subset of the complete SemEval dataset. They haven’t yet extended their evaluations to the entire ATE dataset. Consequently, we have not previously conducted comprehensive experiments on the generative methods. Your inquiries are highly valuable, and we are contemplating the incorporation of a generation approach.\n\n\n**Suggestion #5:** How about using S-BERT? (line 178)\n\n\n**Response to #5:** To simplify this work, we simply employ the embedding layer from a model fine-tuned on the ATE task. We will consider your question positively and explore using other models that focus on similarity recall (e.g. S-BERT) to further improve the accuracy of synonym replacement in our future work.\n\n\n**Suggestion #6:** How obtained? (line 207)\n\n\n**Response to #6:** When we employed ChatGPT to generate ten roles, we did not impose stringent restrictions. Consequently, certain roles may exhibit noise, potentially leading to adverse impacts on results. For instance, the purchasing manager may not be capable of providing valuable information regarding the taste of food, thereby introducing aspect terms unrelated to the review.\n\n\nWe manually observed a relatively good positive effect of the voting rate at 0.7 and 0.8 based on a small dataset taken from the validation set. It is worth noting that an in-depth investigation into the roles’ reliability presents interesting challenges.\n\n\n**Suggestion #7:** Why are the results in Table 3 for BERT-pt-based PST without CHEF different from the results in Table 2 for Wang et al 2021? Isn’t this exactly the same setup? If not, what is the difference? The difference between Table 2 and Table 3 for BERT-pt-based PST is much larger than the improvement you show here from CHEF! (line 252)\n\n\n**Response to #7:** Table 2 shows the performance of PST reported in their paper. Table 3 shows the performance of our reproduced PST.\n\n\nWe were unable to access the comprehensive augmenting data utilized by PST for training. Consequently, we recreated the augmented data by following the procedure outlined in their paper, which may have resulted in some deviations. To accurately reflect the performance improvement achieved by our approach, we carried out the comparison in Table 3.\n\n\n**Suggestion #8:** These improvements are tiny and the question is whether any of them are statistically significant? (line 254)\n\n\n**Response to #8:** We totally agree with your opinion that the improvement on some datasets is insignificant in amplitude. However, our approach simply uses ChatGPT as an auxiliary role without training the model. We will show the statistical significance analysis results in the Appendix and provide all the results in the extended version.\n\n\n**Suggestion #9:** You really should do ablation studies to show which part of CHEF helps (generating paraphrases; notion of perspective; synonym replacement; voting. The reader wants to know why CHEF is helping! (line 257)\n\n\n**Response to #9:** We have meticulously considered your suggestion, and present the results of ablation studies in the Appendix behind these replies. Besides, We intend to provide all the results in the extended version.\n\n\n**Suggestion #10:** This suggests precision is going down. You should always provide both recall and precision in your results so readers can understand what is happening. F1 is just one way of combing R and P, maybe some readers are interested in increasing R only for their problem and they care less about precision. (line 274)\n\n\n**Response to #10:** The provided recall value is derived from instances in which the model made errors in the original sentences. This is intended to demonstrate that our approach can effectively recall more aspect terms.\n\nWe are willing to provide data reflecting changes in recall and precision across all the samples. It shows a decline in precision and a corresponding increase in recall, which could potentially be improved through further enhancements in our post-processing techniques. In Table 3 in the Appendix (behind our replies), we present an overview of the fluctuations in precision (P), recall (R), and F1 scores for our method when applied to BERT-base.\n\n\n\n----------Appendix ----------\n\n\n**Table 1. Ablation Studies**\n\n\n| **Res14**                    | Restaurant Owner      | Chef                  | Waiter            | Restaurant Manager     | Purchasing Manager             | Catering Consultant      | Receptionist     | Financial Officer      | Cleaning Staff     | Diner                      |\n| ---------------------------- | --------------------- | --------------------- | ----------------- | ---------------------- | ------------------------------ | ------------------------ | ---------------- | ---------------------- | ------------------ | -------------------------- |\n| BERT-base                    | 88.03                 | 88.03                 | 88.03             | 88.03                  | 88.03                          | 88.03                    | 88.03            | 88.03                  | 88.03              | 88.03                      |\n| only combine (lines 117-118) | 70.65                 | 61.47                 | 68.82             | 65.51                  | 51.94                          | 60.87                    | 70.14            | 61.06                  | 64.66              | 68.11                      |\n| redundancy elimination       | 87.10                 | 87.67                 | 87.28             | 87.15                  | 86.60                          | 87.19                    | 87.60            | 87.27                  | 87.35              | 87.40                      |\n| synonym replacement          | 87.04                 | 87.45                 | 87.69             | 87.26                  | 87.54                          | 87.22                    | 87.80            | 87.69                  | 87.27              | 87.25                      |\n| multi-role voting            | 88.25                 | 88.25                 | 88.25             | 88.25                  | 88.25                          | 88.25                    | 88.25            | 88.25                  | 88.25              | 88.25                      |\n| **Res15**                    | **Restaurant Owner**  | **Chef**              | **Waiter**        | **Restaurant Manager** | **Purchasing Manager**         | **Catering Consultant**  | **Receptionist** | **Financial Officer**  | **Cleaning Staff** | **Diner**                  |\n| BERT-base                    | 72.17                 | 72.17                 | 72.17             | 72.17                  | 72.17                          | 72.17                    | 72.17            | 72.17                  | 72.17              | 72.17                      |\n| only combine (lines 117-118) | 57.24                 | 47.92                 | 55.64             | 53.48                  | 47.7                           | 49.65                    | 56.68            | 53.72                  | 45.66              | 50.68                      |\n| redundancy elimination       | 71.93                 | 70.59                 | 71.70             | 72.18                  | 70.80                          | 71.67                    | 71.64            | 71.55                  | 71.01              | 71.56                      |\n| synonym replacement          | 71.93                 | 71.00                 | 71.54             | 71.31                  | 70.85                          | 71.53                    | 71.51            | 70.78                  | 70.72              | 70.83                      |\n| multi-role voting            | 72.92                 | 72.92                 | 72.92             | 72.92                  | 72.92                          | 72.92                    | 72.92            | 72.92                  | 72.92              | 72.92                      |\n| **Res16**                    | **Restaurant Owner**  | **Chef**              | **Waiter**        | **Restaurant Manager** | **Purchasing Manager**         | **Catering Consultant**  | **Receptionist** | **Financial Officer**  | **Cleaning Staff** | **Diner**                  |\n| BERT-base                    | 74.88                 | 74.88                 | 74.88             | 74.88                  | 74.88                          | 74.88                    | 74.88            | 74.88                  | 74.88              | 74.88                      |\n| only combine (lines 117-118) | 61.55                 | 54.41                 | 62.66             | 58.75                  | 51.12                          | 55.58                    | 62.56            | 58.09                  | 56.03              | 57.14                      |\n| redundancy elimination       | 73.60                 | 74.32                 | 74.20             | 74.26                  | 73.98                          | 73.90                    | 73.72            | 73.50                  | 73.98              | 74.51                      |\n| synonym replacement          | 74.87                 | 74.04                 | 74.48             | 74.54                  | 74.53                          | 73.81                    | 73.47            | 72.89                  | 74.06              | 74.10                      |\n| multi-role voting            | 75.56                 | 75.56                 | 75.56             | 75.56                  | 75.56                          | 75.56                    | 75.56            | 75.56                  | 75.56              | 75.56                      |\n| **Lap14**                    | **Hardware Engineer** | **Software Engineer** | **Case Designer** | **Test Engineer**      | **Technical Support Engineer** | **Supply Chain Manager** | **Marketer**     | **Purchasing Manager** | **User**           | **Customer Service Staff** |\n| BERT-base                    | 82.96                 | 82.96                 | 82.96             | 82.96                  | 82.96                          | 82.96                    | 82.96            | 82.96                  | 82.96              | 82.96                      |\n| only combine (lines 117-118) | 42.99                 | 42.17                 | 40.93             | 51.41                  | 55.03                          | 45.54                    | 50.33            | 43.65                  | 62.26              | 58.34                      |\n| redundancy elimination       | 82.16                 | 82.32                 | 81.89             | 82.59                  | 81.71                          | 82.25                    | 82.88            | 82.33                  | 82.32              | 81.93                      |\n| synonym replacement          | 82.45                 | 82.13                 | 82.22             | 82.82                  | 82.18                          | 82.53                    | 82.84            | 82.83                  | 82.81              | 82.04                      |\n| multi-role voting            | 83.39                 | 83.39                 | 83.39             | 83.39                  | 83.39                          | 83.39                    | 83.39            | 83.39                  | 83.39              | 83.39                      |\n\n\n\n**Examples**\n\n**1. Example of redundancy elimination:**\n\n**Original**: Better than nobu, tomoe, taka, yama, sugiyama, and the rest.\n\n**Waiter**: Our **chef's sushi** is so amazing, it's better than Nobu, Tomoe, Taka, Yama, Sugiyama, and other well-known sushi spots in the city.\n\n**Ground-truth**: [No Aspect]\n\n**Output from the rewritten sentence**: chef's sushi\n\n\n\n**2. Example of synonym replacement**\n\n**Original**: Best **Indian food** I have ever eaten.\n\n**Chef**: I put a lot of effort into perfecting our **Indian dishes**.\n\n**Ground-truth**: Indian food\n\n**Output from the rewritten sentence**: Indian dishes\n\n\n\n**3. Example of multi-role voting**\n\n**Original**: I have to say they have one of the fastest **delivery times** in the city.\n\n**Chef**: I can prepare the order quickly to ensure timely **delivery**.\n\n**Ground-truth**: delivery times\n\n**Output from the rewritten sentence**: delivery\n\n\n**Table 2: Statistical Significance Analysis**\n| P-value                         | Lap14 | Res14 | Res15 | Res16 |\n| ------------------------------- | ----- | ----- | ----- | ----- |\n| Bert-base VS. BERT-base w/ CHEF | 0.009 | 0.046 | 0.002 | 0.021 |\n| PST VS. PST w/ CHEF             | 0.015 | 0.049 | -     | -     |\n\n\n**Table 3: Performance change**\n| %    | Lap14        | Res14        | Res15        | Res16        |\n| ---- | ------------ | ------------ | ------------ | ------------ |\n| P    | 85.97->84.98 | 88.23->86.82 | 70.28->70.07 | 74.76->74.37 |\n| R    | 80.15->81.85 | 88.85->89.73 | 74.17->76.01 | 75.00->76.80 |\n| F    | 82.96->83.39 | 88.03->88.25 | 72.17->72.92 | 74.88->75.56 |\n\nDue to the long training time of PST, accurate statistical significance analysis results could not be obtained in the short term. We intend to provide all the results in the extended version.\n\n\nThank you again for your insightful feedback, which undoubtedly contributes to enhancing the quality and enriching the final version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The paper presents a novel approach to using ChatGPT for improving performance on an NLP problem, which could be of direct relevance to, or of inspiration to, other researchers, not only those interested in ABSA.\n* Overall clearly written.\n* Great acronym (just kidding).", "reasons_to_reject": "* The way of using ChatGPT is complex (and interesting), but the authors do not present an ablation study to show which steps help.  Update after rebuttal:  the authors provided a table with ablation study data in the rebuttal (thanks!), but the amount of data is overwhelming without a proper interpretation and integration into the paper.  However, I thank the authors for being responsive to the suggestion that several reviewers had.\n* The improvements are small, and in fact smaller than the change in results due to a re-implementation (as confirmed by the authors in rebuttal).  While it is not the standard in our field to report statistical significance, when improvements are really small, it becomes perhaps more urgent to do so.\n* If, as this reviewer believes, aspect Term Extraction (ATE) is only useful as a pre-processing step for ABSA, then the reader would also like to see a comparison with end-to-end systems.", "questions_to_authors": "A) Is ATE useful for something other than ABSA?  If no, have you done experiments comparing with end-to-end systems (presumably generative, such as Zhang et al. 2021)?  This would obviously require integration with an ABSA system which takes the aspect terms as input.  \n\nB) Do you have any data that addresses the question which of the components of your approach actually help?", "paper_topic_and_main_contributions": "This paper presents a system which finds sequences in sentences that are “aspects”, which is a first step in aspect-based sentiment analysis (ABSA).  The innovation consists in using ChatGPT to generate new data.  This is not done simply by generating more training data, but by generating loose paraphrases of an input sentence from the point of view of other participants in the domain, and then aligning aspects in the new paraphrases with the original sentence.  This approach is interesting and original.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "039: ATE is a natural language processing task, which aims to extract aspect terms from sentences (Jakob and Gurevych, 2010).\n\nIs this task performed only in the context of sentiment analysis, or is it also important for other tasks? Maybe make this clear from the beginning.\n\n041: The aspect term refers to a word, phrase or named entity depicting a certain domain-specific attribute.\n\nYou are defining this term here, so put it in quotes or italics,\n\n\n\n045: it appears as the sign of an Italian trattoria.\n\nSign -> Name?\n\n\n\n044: (1) is specified as the restaurant- domain aspect term\n\nDoes the restaurant domain have only one aspect term? If not, say “a restaurant domain…”.\n\n\n\n\n044: span “al di la” in…\n\nHowever, you don’t need both quotes and italics.\n\n\n\n062: For ex- ample, the context “Love” in (1) is uninformative for recognizing “al di la”.\n\nWhy? In the context of sentiment analysis, whatever follows “love” would be as aspect, no?\n\n\n\n153: [S k] is [Content]”.\n\nI don’t understand this. Explain. What is “content”?\n\n\n\n157: Redundant results may be obtained,\n\nOdd choice of word — I would expect a “redundant” result precisely to have occurred before. Maybe use a different word — “extraneous” or “supplemental” or something?\n\n\n\n120: We follow the common practice (Chernyshevich, 2014; Toh and Wang, 2014; San Vicente et al., 2015) to treat ATE as a sequence labeling task.\n\nThis is an old approach — why not use a generative approach?\n\n\n\n178: Note that Cosine similarity is computed over the em- beddings of each pair {u ij,˚u ik}.\n\nHow about using S-BERT?\n\n\n\n207: For the Restaurant domain, a controversial re- sult t iis finally adopted only if v iis no less than a threshold of 0.7.\n\nHow obtained?\n\n\n\n\n227: We respectively use BERT baseand BERT pt-based PST (Wang et al., 2021) to construct the extractors, where PST achieved the best performance so far.\n\n—> “We use BERT base and BERT pt -based PST (Wang et al., 2021)” — you can only use “respectively” if there is a parallel situation to which this conjunction applies.\n\n\n\n252: In Table 3, we report the effects of CHEF on both the BERT baseand BERT pt-based PST, where multi- role voting is used for prediction combination.\n\nWhy are the results in Table 3 for BERT-pt-based PST without CHEF different from the results in Table 2 for Wang et al 2021? Isn’t this exactly the same setup? If not, what is the difference? The difference between Table 2 and Table 3 for BERT-pt-based PST is much larger than the improvement you show here from CHEF!\n\n\n\n\n254: It can be observed that CHEF enables both extractors to achieve better performance on all the test sets, without retraining and fine-tuning.\n\nThese improvements are tiny and the question is whether any of them are statistically significant?\n\n\n\n257: 4.4 Discussion\n\nYou really should do ablation studies to show which part of CHEF helps (generating paraphrases; notion of perspective; synonym replacement; voting. The reader wants to know why CHEF is helping!\n\n\n\n274: The most significant improvement of recall rate reaches about 9.8%. Adaptability– CHEF applies more to short and long sentences.\n\nThis suggests precision is going down. You should always provide both recall and precision in your results so readers can understand what is happening. F1 is just one way of combing R and P, maybe some readers are interested in increasing R only for their problem and they care less about precision.\n\n\n\n279: The former generally contains uninformative con- texts, while the latter contains lower-quality con- texts due to noises.\n\nnoise\n\n\n\nFigure 2 shows the ATE perfor- mance at the original sentences of different lengths, and that at the rewritten cases by CHEF (with multi- role prediction combination).\n\nat —> on"}}]

Author rebuttal:

Meta-review: This paper explores a method for aspect term extraction that leverages ChatGPT to generates higher quality data to facilitate the predictions. The reviewers have several concerns for the paper regarding marginal performance improvement and additional experiments for the contribution of different steps in the pipeline. The rebuttal has provided further experiment results and clarified some details. The authors are encouraged to update the paper according to the rebuttal and discussion with the reviewers.
Decision: Accept-Findings