Reducing Spurious Correlations in Aspect-based Sentiment Analysis with Explanation from Large Language Models
Authors: Qianlong Wang
Keyang Ding
Bin Liang
Min Yang
Ruifeng Xu
Venue: EMNLP 2023 Findings (2023)
Keywords: aspect-based sentiment analysis
spurious correlations
large language models
TL;DR: We use explanations from large language models to reduce spurious correlations in ABSA models.

Abstract: Recently, aspect-based sentiment analysis (ABSA) models have yielded promising results. However, they are susceptible to learning spurious correlations between certain words of the input text and output labels while modeling the sentiment feature of the aspect. This spurious correlation will potentially undermine the performance of ABSA models. One direct solution for this problem is to make the model see and learn an explanation of sentiment expression rather than certain words. Motivated by this, we exploit explanations for the sentiment polarity of each aspect from large language models (LLMs) to reduce spurious correlations in ABSA. First, we formulate a prompt template that wraps the sentence, an aspect, and the sentiment label. This template is utilized to prompt LLMs to generate an appropriate explanation that states the sentiment cause. Then, we propose two straightforward yet effective methods to leverage the explanation for preventing the learning of spurious correlations. We conducted extensive comparative experiments on five datasets by integrating them with some representative ABSA models. Results show that our methods can achieve performance gains and enhance the performance and generalization ability of ABSA models.

Reducing Spurious Correlations in Aspect-based Sentiment Analysis with Explanation from Large Language Models
Qianlong Wangh 12 Keyang Ding Bin Liang424, Min Yang', Ruifeng Xu
Harbin Institute of Technology, Shenzhen, China
*Guangdong Provincial Key Laboratory Novel Security Intelligence Technologies 'Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences *The Chinese University of Hong Kong SPegg Cheng Laboratory, Shenzhen, China qlwang1 50outlokk com, keyang ding@stu. hit. edu. cn, bin. liang@cuhk, eeraanuuuk.edu...... hk, min.yang@siat .n.cn, xurui feng@hit..... edu. cn
Abstract
Training Samples
Label
company provides UPS [shipping], fast, great!
positive
Recently, aspect-based sentiment analysis (ABSA) models have yielded promising results. However, they are susceptible to learning spuri- ous correlations between certain words of the input text and output labels while modeling the sentiment feature of the aspect. This spuri- correlation will potentially undermine the performance of ABSA models. One direct so- lution this problem to make the model see and learn explanation of sentiment expres- sion rather than certain words. Motivated by this, we exploit explanations for the sentiment polarity of each aspect from large language models (LLMs) to reduce spurious correlations in ABSA. First, we formulate prompt tem- plate that wraps the sentence, an aspect, and the sentiment label. This template utilized to prompt LLMs to generate an appropriate expla- nation that states the sentiment cause. Then, we propose two straightforward yet effective meth- ods to leverage the explanation for preventing the learning of spurious correlations. We con- ducted extensive comparative experiments on five datasets by integrating them with some rep- resentative ABSA models. Results show that methods achieve performance gains and enhance the performance and generaliza- tion ability ABSA models.
the [Final Cut Pro on this laptop is andtland easy.
positive
super fast proocesoor) and really nice graphics card.
positive
Testing Sample
the [hattery life) was faster than expected. (prediction: positive)
negative
Figure 1: Examples of the spurious correlation between context words "fast" and label "positive" in the training samples. This spurious correlation does not hold for the testing sample. Aspect terms are marked parentheses.
such linguistic features (Negi and Buitelaar, 2014). However, feature engineering could be a time-consuming process, requiring significant ef- fort and expertise. To solve this dilemma, deep learning solutions are utilized to address ABSA due to their powerful contextual feature modeling capa- bility. From conventional neural networks (Ruder et al., 2016; Xue and Li, 2018; Ma et al., 2018) to attention mechanisms (Tang et al., 201 Li et al., 2018a; Gu al., 2018; Fan et al., 2018), these solutions focus on modeling the dependency rela- tionship between an aspect and its corresponding opinion expressions. With the emergence of fine- tuning paradigm, the attention mechanism armed with pre-trained language models (PLMs) (Devlin et al., 2019; Song et al., 2019; Wang et al., 2020; Tian et al., 2021; Nazir et al., 2022: Zhang et al.. 2022) further strengthens the connection between the aspect and its context.
1 Introduction
Aspect-bas based sentiment analysis (ABSA) aims to identify the sentiment polarity (e.g., positive, neu- tral, and negative) of a specified aspect in a review (Pontiki et al., 2014). For example, given review "great food but the service was dreadful!" and two aspects "food" and "service". this task needs infer their sentiment polarities "positive" and "negative", respectively.
Despite their satisfactory results, most neural net- work methods may indulge in learning statistically spurious correlations while modeling the sentiment feature of aspect on the context. Here, spurious correlation (Wang and Culotta, 2020; Wang et al., 2022b) refers to the dependence of the model on
Traditional ABSA methods primarily rely on machine learning techniques, which incorporate some handcrafted features to enhance performance,
Corresponding author.

---

certain words of the input text without deeper un- derstanding of the contextual semantics, which has a know-ii-when--yuu-see character. Taking the ex- ample in Figure for illustration, the opinion word "fast" expresses different sentiment polarities of the aspect terms in distinct contexts. Here, 92% of as- pect sentiment is "positive" in the training samples when counting the proportion of aspect sentiment polarity that co-occurs with "fast". Due to this unbalanced distribution, in the training phase, the neural models assume that there is a strong cor- relation between "fast" and "positive", especially for short texts. Consequently, when faced with a testing sample containing a derivative "faster". the trained models will predict the incorrect sentiment label "positive" based on this spurious correlation learned superficially before. Thus, most neural models may encounter difficulty in navigating spu- rious correlations because of shallow understand- ing. Besides, they lack the capacity to self-correct, resulting in undermining their effectiveness and generalization.
to provide a relevant explanation according the context. In this way, the output explanation can provide the reason for sentiment and may contain some external knowledge thanks to the powerful ca- pabilities of LLMMs Then, we propose two methods to employ this explanation to improve the effective- ness and generalization of ABSA models. One is the augmentationn-ausel method, which directly treats these explanations containing the aspect term as training samples. We mix these explanations with the original training samples to train more robust ABSA model. This method can not only re- lieve the statistical bias in original samples but also learn a range of sample patterns. The other is the distillation- -ased method, whose basic idea is to distill the knowledge embedded in the explanation into student ABSA model. By the distillation loss, the student ABSA model can mimic the two behav- iors of the teacher, i.e., sentiment representation and output logit. this way, the explanation can guide the learning of ABSA models and prevent them from focr-sog on spurious correlations. In summary, our contributions are as follows:
One straightforward solution to alleviate the spu- rious correlation problem is to make models attend to an explanation of sentiment expression rather than certain words in the context. Here, explana- tion refers to the reasons for the sentiment polarity of aspect term obtained by deeply understanding the contextual semantics. However, for each train- ing sample, is a tricky problem to derive the senti- ment explanation given the aspect and its sentiment. Recently, large language models (LLMMs) (Brown et al., 2020) have achieved remarkable success in a wide range of NLP capabilities, including genera- tion and contextual understanding, In addition, they are knowledgcable due to the substantial linguis- tic (Liu et al., 2019) and factual world knowledge learned. Thus, LLMs can be exploited to gener- ate an explanation toward the sentiment of aspect through prompt-driven contextual understanding (Bian et al., 2023). Taking the second training sam- ple in Figure 1 as an example, LLMs can yield an explanation, "The sentiment towards 'Final Cut Pro" is positive because the speaker praises its efficiency and fren-friendliness on the laptop, in- dicating satisfaction and favorable feelings about the software."
To our knowledge, we are the first to induce LLMs to generate an explanation for the as- pect's 's sentiment and use it to reduce spurious correlations in the ABSA task.
We devise two straightforward methods for utilizing this explanation, which can be inte- grated into most mainstream baselines.
We conduct extensive experiments on five benchmark datasets, showing that baselines armed with the proposed methods can achieve better performance on inference and general- ization.
2 Related Work
Aspect-based Sentiment Analysis. ABSA aims to identify the sentiment polarity of each aspect mentioned the text. To solve this task, vari- ous neural networks with the attention mechanism are utilized to find the semantic relation of an as- pect and its context for capturing the corresponding opinion expression (Tang et al., 2016b; Ma et al., 2017: Li al., 2018c; Fan al., 2018: Tan et al., 2019). For instance, Fan et al. (2018) exploited a multi-grained attention mechanism to capture the word-level interaction between the aspect and its relevant context. The idea behind the attention
Inspired by this, we leverage explanations from LLMs to reduce spurious correlations ABSA. Specifically, we first design a prompt containing an aspect term and its sentiment to induce LLMs

---

step 1. explanation generation
In the following sentence, "the Final Cut Pro on this laptop is fast and easy," explain why the sentiment expressed by aspect term "Final Cut Pro" is "positive"? Limit to forty words. Copying ectiivees from original sentence is not allowed.
Explanation X
The senliment lowards 'inal Cut Pro' positive because the speaker praises its efficiency and user friendliness on the laptop, indicating satisfaction and fevorable foelings about the software.
Sentence
Prompt Template
Final
this
Model
laptop so last and easy. A Aspect Final Cut Pro Labely positive
step 2. explanation exploitation
cts
cts
ats
student
teacher
classifier
classifier
classifier
Enid
encoder
encoder
EMA
encoder
(0, a00f u (08 a1
a0 Y.
( R 20
augmentation-basel based method
distillation based method
Figure 2: The overview proposed framework. This framework consists two steps: explanation generation and explamation exploitation.
mechanism is to focus on the context related to the aspect and shield the irrelevant context. To further pursue this idea, some studies (Song et al., 2019; Li et al., 2020; Yan et al., 2021; Wang et al., 2022b,au applied pre-trained models (PLMs) such as BERT (Devlin al., 2019) to model the semantic rela- tionship between the given aspect and its context. The internal multi-head self-attention mechanism in PLMS more efficient than conventional atten- tion techniques (Vaswani et al., 2017). As a result, these studies consistently delivered better results.
focus on certain words the context to prevent being trapped in the spurious correlation trap.
Large Language Models. With the advent GPT-3 (Brown et al., 2020), LLMs break into the limelight and draw enormous attention. They typ- ically feature a vast array of model parameters and undergo training immensely large volumes of raw data. By learning from data, they memo- rize and understand vast amounts of knowledge (Li et al., 2022) and learn to reason (Wei et al., 2022). Knowledge and reason are crucial for build- ing satisfactory NLP system that can understand and generate human-like language. Consequently, LLMss like ChatGPT can achieve substantial perfor- mance improvements in wide range of NLP tasks, including inference and dialogue, by profoundly comprehending the contextual semantics. Inspired by this, for the sentiment polarity of each aspect, we here apply LLMs to generate an explanation to explain its causes.
Another research trend is to leverage syntactic knowledge from syntactic trees to handle ABSA. This syntactic knowledge helps establish con- nections between the aspect and opinion words and learn syntax-aware feature representations of the aspect (He et al., 2018; Sun et al., 2019; Phan and Ogunbona, 2020; Wang et al., 2020; Tian et al., 2021: Liang et al., 2022). The core idea of these studies is transform constructed syntax depen- dency tree into a graph for posing greater attention important words.
Our Approach
Although these methods obtained promising re- sults by modeling semantic relationships between aspects and contexts, they inevitably plagued by statistical spurious correlations. Unlike them, in this paper, we aim reduce spurious correla- tions with explanations from LLMs. These expla- nations can serve to guide ABSA models not to
3.1 Problem Definition
Given an ABSA training set, each sample consists of a sentence X, aspect a, and sentiment label y. Here, the aspect is sub-sequence token in the sentence. ABSA aims to learn a sentiment classifier that can precisely predict a sentiment polarity y E

---

use them (positive, negative, neural) for each aspect term according the semantics of the sentence."
adjectives from the original sentence is not allowed.
3.2 Overview
We can see that this prompt consists of three components: task description, training sample, and output limitation. They describe the task precisely and form good output guide, which helps to en- hance the generation performance. As shown in the example in Figure 2, LLM tasked with gener- ating a friendly explanation X for the aspect senti- ment. This explamation not only explains why the sentiment occurs based on contextual semantics (i.e., "user-friendliness") but also includes some background knowledge (i.e., "software").
As shown Figure 2, our framework consists of two steps. The first step is explanation generation. Here, for each training sample, we use a prompt template to encapsulate the sentence, the aspect, and its sentiment to drive the LLMs to generate an explanation to indicate the corresponding sentiment cause. The second step is explanation exploitation. Here, we propose two simple yet effective methods exploit explanations for alleviating the spurious correlations in ABSA.
3.3 Explanation Generation
3.4 Explanation Exploitation
Spurious correlations are common current ABSA models, particularly in cases of over- parameterization or insufficient training data (Sagawa et al., 2020). The fundamental reason is that these models might learn statistical corre- lations between superficial textual cues and senti- ment labels rather than achieving profound com- prehension of contextual semantics. Consequently, this problem will hurt the performance and gener- ality of the ABSA classifier.
The explanation generated by the LLM provides with comprehensive view of the original text from the perspective of contextual semantics and back- ground knowledge. Furthermore, the explanation does not have high-frequency adjectives (e.g., fast) due to the limitation in the prompt, which further provides sufficient condition mitigate statisti- cal spurious correlations. Thus, aid the learning of ABSA models and improve the performance of the model. Here, we present two straightforward and model-agnostic methods to achieve this.
In this work, we try to reduce spurious correla- tions in ABSA using explanation. To achieve this, we expect explanation to have two functions: (i) motivating ABSA models infer the aspect's senti- ment by understanding the context rather than some surface words. (ii) providing additional knowledge as background information for better contextual understanding, especially for short texts; Recently, LLMs such as ChatGPT have exhibited incredible contextual understanding and knowledge inference on wide range of NLP (Wei et al., 2022). It inspires to leverage LLMs to generate expla- nation for the aspect's sentiment in each training sample, which has not been explored in the litera- ture. To this end, we design a prompt template to trigger the understanding and inference ability of LLMs, which wraps the sentence X, an aspect and its sentiment
Augmentation-basse Method. In sense, the explanation can be considered as a paraphrase the original sentence, which has the same seman- tic meaning and label but a different description. This different description not only facilitates the alleviation of statistical bias in the original sen- tences but also diversifies the expression of the same sentiment. Thus, mixing the original train- ing data {(X,, aj 31) j with their explanations (X,, 0z 3/) JN can allow for training a more ro- bust ABSA classifier:
2N Σ CE
L.s 2N
(1)
In the following sentence X, explain why the sentiment expressed by aspect term is y. Limit to forty words. Copying
where P(X', .) is the predictive probability distri- bution of sentiment; X' can be either X or X; C' denotes the cross-entropy loss function.
FFor sentences with multiple aspects, we treat other non- targeted aspects as normal context tokens when focusing on the target aspect. In other words, sentence will be processed multiple times, which is equal to the number of aspects it contains.
The explanation is more effective than conven- tional data augmentation methods (Wei and Zou, 2019) 9) as it interprets the contextual semantics and contains some knowledge.

---

Distillation-based Method. Direct mixing can- not align the original sentence with the correspond- ing explanation, which results trouble providing customized guided learning. To this end, we use a guidance strategy to encourage ABSA models to reduce spurious correlations in fitting each sample. This strategy can be viewed as knowledge distil- lation (Hinton al., 2015), which aims to leverage the teacher guide the student's training with the help of explanations. To achieve this guidance, we here make the student model mimic two behaviors the teacher one via the following two losses:
Experiment
4.1 Datasets and Settings
Datasets. We use five benchmark datasets to eval- uate the proposed methods: Lapl4 and R from Pontiki al. (2014), Rest15 from Pontiki et al. (2015), Rest16 from Pontiki and MAMS from Jiang et (2019). All datasets only involve three sentiment labels, positive, neu- tral, and negative. Each sample in these datasets is annotated with aspects and their corresponding sentiment polarities. Here, we adopt the official data splits done in the original papers. The basic statistics are shown in Table 1.
(2016),
Cais N
(L (x)
Dataset
#Pos
#Neu
#Neg
Total
Lap14
train
994
464
2,328
N Σ MS:( 1=1
test
341
169
Ctid
train
2,164
637
807
3,608
N
Rest14
test
728
196
196
1,120
train
912
36
256
1,204
Rest
where (x(X), a) and gx(X%, a) (li; and x)) re- fer to the logits (hidden states), which come from student and teacher networks, respectively; KL denotes the Kullback-Leibler divergence loss func- tion; MSE denotes the mean squared error loss function. By two losses, the explanation is utilized facilitate the learning process of ABSA mod- els and mitigate overly concentrated on shorteut features.
test
326
34
182
542
train
1,240
69
439
1,748
Rese16
test
469
30
117
616
train
3,380
5,042
2,764
11,186
MAMS
dev
403
604
325
1,332
test
400
607
329
1,336
Table I: The detailed statistics of datasets.
Settings. If not otherwise specified, we use Chat- GPT and the pre-trained uncased BERT-base" LLM and encoder in the framework*. respectively. For the classifier, the weight matrix is randomly initialized by a uniform distribution. To avoid over-fitting we apply the dropout (Srivastava et al., 2014) with a probability of 0.1. Besides, we also replace the label words (i.e., positive, neutral, and negative) in the explanation with [MASK] token. We employ the Adam optimizer to optimize pa- rameters. The epoch, batch size, learning rate, and smoothing coefficient are 8, 24, 3e-5, and 0.95, respectively. We limit the maximum length of the token sequence 256.
To yield better guidance, the teacher network tracks an exponential moving average (Tarvainen and Valpola, 2017) of the student network weights. At the time step f, the parameters of the teacher 0 are updated follows:
0. = A- 0, X)
(4)
where & represents
parameters in the student network at time step t; À is smoothing coefficient. With this moving average, the teacher's output is more reliable.
3.5 Training and Testing
We run the experiments five times with ran- dom initialization and report the averaged results. The accuracy (Acc.) and averrge Fl (F1) scores are used as the evaluation metric.
For the augmentation-baut bassed method, we train the parameters the ABSA model directly by optimiz- ing Ldts (Eq. 1). For the distillation- based method, we update parameters the student model by opti- mizing the sum of Lds (Eq. 1), Lais (Eq. 2), and Crid (Lqq. 3). In the test phase, the sentence and aspect fed into the student network to predict the label.
4.2 Baselines
To evaluate the effectiveness and generalization of the proposed methods, we integrate them with
'ht Ppttps:/ /gg htt/gggttuu..... htt.u.cco//goo///// hwm/google-rrsearch/beeee
II this work, the teacher model and the student model have the same framework.
"The proposed framework only loads the I.I.M for infer- ence without involving training.

---

Datasets
Lap14
Rest14
Rest
Restl6
MAMS
Models
F1
Acc.
Fl
F1
Acc.
F1
F1
Large Language Models
MOSS (zcro-shot)
68.34
51.18
75.89
52.22
81.55
58.40
85.06
58.35
41.92
36,21
MOSS (few-shot)
70.85
61.21
77.59
61.81
82.29
59.04
87.18
64.04
43.71
42.43
ChatGPT (zoro-shot)
78.64
70.70
80.39
71.33
77.01
63.21
83.39
68.75
63.67
64.08
ChatGPT (few-shot)
78.90
75.61
83.74
77.08
82.80
71.31
84,83
69.90
60.18
60.57
the PI.Ms-bosed Models
BERT
75.80
71.67
82.59
74.10
80.97
63.52
88,47
71.16
82.68
82.37
augmentation
77.74
73.11
84,29
76.07
82.03
65.80
89.96
73.06
84.51
84.11
distillation
78.68
75.19
84.66
76.18
82.63
65.97
90.12
73.69
83.68
83.38
BERT
T7.59
73.14
84.16
76.62
82,83
65.81
91.73
73,93
83.48
83.10
augmentation
77.90
74.66
85.71
78.37
83.50
67.48
92.75
75.48
83.86
83.73
distillation
77.54
73.71
85.96
78.93
84.14
68.05
92.02
74.45
84.43
83.92
the Attention-busee Models
TNet
76.63
72.00
82.68
74.33
81.55
64.99
88.80
71.41
82.84
82.42
augmentation
77.82
73.24
84.30
75.96
82.48
66.28
90.12
73.22
83.73
83,15
distillation
78.04
73.32
84,75
76.68
82.97
66.17
90.30
73.81
83,98
83.42
AEN
77.27
72.68
83.66
76.02
82.00
67.27
89.45
73.61
82.86
82.58
augmentation
78.37
74.25
85.54
79.58
83.03
69.85
90.42
75.04
83.34
83.03
distillation
78.40
73.96
84.20
77.20
82.47
67.84
91.05
75.45
83.19
82.70
the Graph- bused Models
RGAT
77.45
72.70
86.02
80. 74
81.80
68.21
89.51
75.81
82.93
82.75
augmentation
80.31
75.73
87.45
82.41
83.86
70.42
91.61
77.44
84.61
84.03
distillation
78.11
74.06
86.30
81.12
82.55
69.29
90.47
77.05
83.78
83.18
GualGCN
81.62
74.67
85.20
80.16
82,33
68.12
90.91
77.86
83.83
83.47
augmentation
81.56
75.92
18
80.50
83.98
70.86
91.12
77.97
84,28
83,
distillation
81.22
75.43
86.37
80.63
82.69
69.43
91.45
78.12
84.68
84.23
Table Main experimental results (%) five ABSA datasets. The zero-shot and few-shot indicate that LLM uses O and 4 demonstrations in the in-context learning paradigm (Dong et al., 2022), respectively. All the results are the average five seeds. For each baseline, the best scores are bold.
some representative ABSA models and compare performance. These ABSA models could be cat- egorized into three groups. (1) the PLMs-based models, which includes BERT (Devlin al., 2019) and BERT-PT (Xu et al., 2019). (2) the attention- based models, which includes TNet (Li et al., 2018b) and AEN (Song et al., 2019). (3) the graph- based models, which includes RGAT (Wang et al., 2020) and DuaLGCN (Li et al., 2021).
First, ABSA models equipped with our meth- ods (i.e., augmentation and + distillation) achieve better performance than peer competi- tors both accuracy and F1. Among them, the bigger improvements in accuracy and F1 are 2.88% (BERT+distillation on the Lapl4) and 3.56% IAEN+augmentation on the Rest14), respec- tively. These improvements show that (i) the pro- posed explanation can effectively mitigate spurious correlations, and (ii) our methods can be seamlessly compensated to existing ABSA models.
In addition the ABSA models mentioned above, we also introduce LLMs (MOSS and ChatGPT as strong competitors.
Second, the graph- ABSA models perform better than the attention based ones. For exam- ple, DualGCN improves performance by 3.99% in accuracy and 3.67% in F1 over TNet on the Lap14. Although the graph-based models have ob- tained satisfactory results, of 0.28~2.86%% in accuracy and 0.38~3.03%% in Fl when integrated with our methods. It indicates that while exploiting the syntactic knowledge con- necting aspects and opinion words to improve per- formance, they may still model shortcut features
4.3 Main Results
Table shows the experimental results of ABSA models on five datasets. We can draw the following conclusions from this table:
observe boost
'The snapshot version parameters the MOSS MOSS-moonn-03--s
16B, respectively. Please refer https://moss. fastnll. top/
"The snapshot version and parameters of the ChatGPT are texi-davinci-100 and 175B, respectively. Please refer to https://openai. hwtp:/pppaaa..ooao/ooooooaaa/oaa www.//////////o//ccc/ccc//oooolo chattpt.

---

Datasets
Lap14
Rest14
R
Restl6
MAMS
Models
Acc.
F1
Acc.
F1
F1
Acc.
F1
Acc.
F1
BERT
73.80
71.67
82.59
74.10
80.97
63.52
88,47
71.16
82.68
82.37
+ SWAP
77.03
72.33
83.24
75.15
81.32
64.68
89.58
72.42
82.76
83.00
+ ADD
77.21
72.46
83.80
75.71
81.55
64.81
89.10
72.59
83.31
82.85
DELETE
76.84
72.56
82.95
73.22
80.63
63.02
89.26
71.57
83.03
82.88
MASK
76.65
72.64
83.57
74.70
79.15
60.50
89.12
71.58
83.06
82.80
TRANSLATION
77.27
72.80
84.08
74.90
81.52
65.18
89.49
72.75
82.91
83.07
augmentation
77.74
73,11
84,29
76.07
82,03
65,80
89,96
73.06
84,51
84.11
distillation 78.68
75.19
84,66
76.18
82,63
65.97
90.12
73.69
83,68
83,38
Table 3: Comparison of our methods with five popular data augmentation baselines on the ABSA task. SWAP: randomly swap two tokens; ADD: randomly insert some sampled tokens; DELETE: randomly remove some tokens; MASK: first replace some tokens with [MASK] token and then use BERT to complete the mask language task (Devlin et al., 2019);; TRANSLATION (Sennrich et al., 2016): first translate the into Chinese and then translate the output into English. The best scores are in bold.
because of shallow understanding of some words. Third, LLMs can yield impressive results using few demonstrations. Compared with PLMs, they are scaling up in depth and width. It causes them to become increasingly computationally and storage- intensive, making deployment difficult. This is why we leverage explanations from LLMs to reduce the spurious correlations in ABSA rather than using them directly to solve it.
actually is in used datasets? To answer this ques- tion, we conduct a simple probe experiment on the Aspect Robustness Test Set (ARTS) (Xing et al., 2020). ARTS enrich the initial test sets from Lap14 and Rest by employing three adversarial strate- gies?. Here, if model predicts the same sentiment labels for an original sample as well its adver- sarial samples (their true labels are different), we will assume that there is a spurious correlation be- tween certain contextual words and the predicted label for this original sample. In other words, the predicted label does not change with the contextual semantics because the model only focuses on cer- tain words. Based on this ussumption, we count the percentage of original samples in the test set that contain spurious correlations." 8 According Table 4, we see that spurious correlations do exist in the Lap14 and Rest14 datasets. Moreover, we can observe that the proposed methods reduce the percentage of original samples containing spurious correlations. This may suggest that the generated explanations can alleviate the spurious correlations problem in the ABSA task.
Fourth, we find that the augmentation based method and distillation-based one could not tell who wins and who loses. Each has its own ad- vantages and merits. For example, although the distillation-base based method yields higher results than the augmentation-bass method in some cases, the latter is superior with respect to efficiency. In ad- dition, the augmentation-based method is more ap- plicable in different ABSA models. Therefore, we will subsequently focus more on the augmentation- based method for future research.
Lap14
Rest14
BERT
14.8
14.5
+ augmentation
12.7
11.6
+ distillation
13.0
11.4
AEN
16.3
16.4
+ augmentation
15.2
14.8
Comparison with Data Augmentation Baselines. This work aims to exploit the explanation from LLMs to reduce spurious correlations, which could be viewed as augmented instance of the original sentence. To evaluate its effectiveness, we compare the proposed methods with five data augmentation
distillation 15.3
15.0
Table 4: The proportion (%) of spurious correlations present in the dataset. The lower the better.
5 Discussion
Percentage of Spurious Correlations in the Dataset. In this work. spurious correlation (Wang et al., 2022b) refers to the dependence of the model on certain words in the input text without a deeper understanding of the contextual semantics. A ques- tion naturally arises how much of the correlation
"They are (1) reversing the original sentiment the tar- geted aspect; (2) reversing the sentiment of the non-targeted aspects; and (3) generating more non- non-targeted aspects with opposite sentiment polarities from the targeted aspect.
'It is worth reminding that this percentage is not the actual percentage of spurious correlations in the dataset, which is only an estimate under this assumption.

---

Datasets
I
K
78
Models
Acc.
F1
BERT
77.02
63.83
73,82
68.74
augmentation
78.93
66.33
76.33
72.02
+ distillation
79.11
64.93
75.55
70.99
BERT Faugmentation +distillation
BERT+PT
78,82
70.74
74.35
68.62
+ augmentation
81.25
71.42
75.55
69.91
+ distillation
82.68
75.37
73.20
66.53
10%
100% (a) On Lap Lap14 dataset.
parcenlaged of trining data
Table 5: The generalization results of ABSA models. L + R (or R = L) refer to that the model is trained on Lapl4 (or Rest14) training data and then tested on Restl4 (or Lap14) test data. The best scores for each baseline are in bold.
78
F1
64
57
taugmentation
+disillation
50
baselines." Table 3 reports the experimental results. It can be seen that our methods perform better than all baselines, achieving the biggest improvements 3.48% and 5.47% accuracy and FI, respec- tively. This shows that the explanation from LLMs is more effective because of including not only a contextual understanding of sentiments but also external knowledge. Besides, we find that these augmentation baselines often consistently improve the performance of BERT, showing that modifying the original text may bring gains, despite the noise.
10%
20%
30%
40%
30%
60%
70%
80%
90%
100%
perpentage traiming data
(b) On Rest14 dataset.
Figure 3: Experiments in low-resource scenarios. We limit the percentage training data when fine-tuning.
centage increases before the training size surpasses 50%, indicating that the more training data, the better the model is trained. Nevertheless, upon surpassing this point, the performance fluctuates moderately.
Generalization Analysis. We evaluate the pro- posed methods in the cross-domain scenario to check their effectiveness on generalizability. The experimental results are presented in Table 5. We observe that: (1) Our methods significantly enhance the generalization ability of the peer base- the
DERT
BBERT Aongmentatinn
76
ediidliation
F1 (1/0
85
58
(0,20)
(20,40)
[40, 60)
(0,20)
(00,40) (40,60) toxt length
lines by a substantial margin. We attribute explanations of aspect sentiment that can reduce the spurious correlations in ABSA. (2) Overall, the augmentation method is more effective than the distillation one. A possible reason for this is that explanations containing external knowledge are directly involved in the training, allowing the learning of better transfer features.
text length
(a) On Lap14 dataset.
(b) On Restl dataset.
Figure Length-wise performance comparison. Here, [20, 40) indicates that the token number of the sample satisfies the condition of greater than or equal to 20 and less than 40. The other meanings are similar.
Length-wise Performance Analysis. Spurious correlations are prone to occur when predicting the short text as the model tends resort the learned statistical bias facing a low-informative context. Here, we test length-wise performance to reveal the noticeable advantages of the proposed methods on short texts. Figure 4 provides the test results. We can see that the proposed methods significantly improve the performance of BERT, especially on short texts. It shows that the explanation provided by LLMs for the training samples motivates the ABSA model to trained effectively, thus allow- ing for a better contextual understanding of short texts at testing.
Effectiveness in Low-Resource Scenario. Here, we carry out an experiment to observe the perfor- mance improvements achieved by our proposed methods in low-resource settings. To this end, we vary the percentage of training data from 10% to 100% in increments of 10% and depict results in Figure We can see that: (1) Overall, being armed with our methods can improve the performance of BERT. shows that introducing an explanation for sentiment is useful in low oowrresource scenarios. (2) The performance gradually improves as the per-
"Wo add augmented versions directly to the existing set.
Thus, the training set size will be doubled after this operation.

---

The Appendix has more discussion and anal- ysis, i.e., Quality of the ainoomatccaly-Geneaaue Explanations, Effect Different Prompt Templates on Performance, and Error Analysis.
Acknowledgements
This work was partially supported by the National Natural Science Foundation of China (62006062, 62176076), Natural Science Foundation of Guang- dong 2023A151501 Shenzhen Foundational Research Funding JCYJ202103244 15614039 and JCY1202201180222 15002, Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies 2022B 12 20100005.
6 Conclusion
this paper, we introduce an effective two-step framework to mitigate spurious correlations in ABSA. First, we formulate prompt template to in- duce LLMs to generate an appropriate explanation that states the sentiment cause. Subsequently, we propose two straightforward methods that utilize the generated explanation to prevent the assimila- tion of spurious correlations. Our comprehensive experiments on five ABSA datasets show that base- lines armed with our methods outperform peers in prediction performance and generalization.
References
Ning Bian, Xianpei Han, Le Sun, Hongyu Lin, Yaojie Lu, and Ben He. 2023. Chatgpt is a knowledgeable but inexperienced solver: An investigation of com- monsense problem in large language models. arXiv preprint arXiv:2303.1642. 16421.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models few-shot learners. NeurIPS, 33:1877-1901.
Limitations
In this section, we list two limitations to understand this work more comprehensively:
Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and
Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In NAACL, pages 4171-4186.
1.
The prompt template designed in this work consists of three components: the task descrip- tion, the training sample, and the output limi- tation. Generally speaking, prompt-rich tem- plate allows LLMs to generate more helpful explanations about sentiment and richer rele- vant external knowledge. In this work, we did not design prompt-rich template because this manual design process is time-consuming and cumbersome. In addition, designing com- plex and information-rich prompt template is not the research focus of this work.
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiy- ong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey for in-context learning. arXiv preprint arXiv:2301 10034..
Feifan Fan, Yansong Feng. and Dongyan Zhao. 2018. Multi-grained attention network for aspect-level sen- timent classification. In EMNLP, pages 3433-3442.
Shuqin Gu, Lipeng Zhang, Yuexian Hou, and Yin Song. 2018. A position-aware bidirectional attention net- work for aspect-level sentiment analysis. In COL- ING, pages 774-784.
2. We leverage explanations to reduce spuri- ous correlations in the ABSA task. In this work, we generate an explanation for the sen- timent label in each training sample, which subsequently participates in the model train- ing process. Although spurious correlations are caused by statistical bias during training, not all training samples bring bias interference to the model. Therefore, the participation of all explanations in model training is exten- sive operation, which somehow results in waste of training resources. How to identify whether a training sample potentially brings spurious correlation interference can a di- rection for subsequent research.
Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier, 2018. Effective attention modeling for aspect-level sentiment classification. In COLING, pages 1121-1131.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in neural network. arXiv preprint arXiv: 1503.02531.
Qingnan Jiang, Lei Chen, Ruifeng Xu, Xiang Ao, and Min Yang. 2019. A challenge dataset and effec- tive models for aspect-based sentiment analysis. In EMNLP, pages 6280-6285.
I Lishuang Li, Yang Liu, and AnQiao Zhou. 2018a. Hier- archical attention-based position-aware network for aspect-level sentiment analysis. In COLING, pages 181-189.

---

Ruifan Li, Hao Chen, Fangxiang Feng, Zhanyu Ma. Xiaojie Wang, and Eduard Hovy. 2021. Dual graph convolutional networks for aspect-based sentiment analysis. In ACL, pages 6319-6329.
Maria Pontiki, Dimitrios Galamis, Harris Papageorgiou. Suresh Manandhar, and lon Androutsopoulos. 2015. Semeval-2015 task 12: Aspect based sentiment anal- ysis. In SemEval, pages 486-495.
Xiang Lorraine Li, Adhiguna Kuncoro, Jordan Hoff- mann, Cyprien de Masson d' Autume, Phil Blunsom, and Aida Nematzadeh. 2022. systematic investiga- tion of commonsense knowledge in large language models. In EMNIP, pages 1838-11885 1855.
Maria Pontiki, Dimitrios Galanis, John Paylopoulos, Harris Papagcorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: As- pect based sentiment analysis. In SemEval, pages 27-35.
Xin Li, Lidong Bing. Wai Lam, and Bei Shi. 2018b. Transformation networks for target-oriented senti- ment classification. In ACI, pages 946-956.
Sebastian Ruder, Parsa Ghaffari, and John Breslin, 2016. A hierarchical model of reviews for aspect- based sentiment analysis. In EMNLP, pages 999- 1005.
Xin Li, Lidong Bing, Piji Li. Wai Lam, and Zhimou Yang. 2018c. Aspect term extraction with history at- tention and selective transformation. In IJCAI, pages 4194-4200.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. 2020. An investigation of why overpa- rameterization exacerbates spurious correlations. In PMLR.
ICML. pages
Xinlong Li, Xingyu Fu, Guangluan Xu, Yang Yang, Jiuniu Wang, Li Jin, Qing Liu, and Tianyuan Xiang. 2020. Enhancing bert representation with context- aware embedding for aspect-based sentiment analysis. IEEE Access, 8:46668-46876.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In ACL, pages 86-96.
Bin Liang, Hang Su, Lin Gui, Erik Cambria, and Ruifeng Xu. 2022. Aspect-based sentiment anal- ysis via affective knowledge enhanced graph con- volutional networks. Knowledge-Band Based Systems, 235:10
Youwei Song, Jiahai Wang, Tao Jiang, Zhiyue Liu, and Yanghui Rao. 2019. Attentional encoder network for targeted sentiment classification. arXiv preprint
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-195)
Nelson F Liu, Matt Gardner, Yonatan Belinkov, Matthew E Peters, and Noah A Smith. 2019. Lin- guistic knowledge and transferability of contextual representations. In NAACI, pages 1073-1094.
Dehong Ma. Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017. Interactive attention networks for aspect-level sentiment classification. In IJCAI, pages 4068 4074.
Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao. and Xudong Liu. 2019. Aspect-level sentiment analy- sis via convolution over dependency tree. In EMNLP, pages 5679-5688.
Yukun Ma, Haiyun Peng, Tahir Khan, Erik Cambria, and Amir Hussain. 2018. Sentic Istm: A hybrid network for targeted aspect-based sentiment analysis. Cognitive Computation, 10:639-650.
Xingwei Tan, Yi Cai, and Changxi Zhu, 2019. Recog- nizing conflict opinions in aspect- level sentiment clas- sification with dual attention networks. FMNLP, pages 3426-3431.
Ambreen Nazir, Yuan Rao, Lianwei Wu, and Ling Sun. 2022. laf-Ig: interactive attention fusion network with local and global perspective for aspect-based sentiment analysis. IEEE Transactions Affective Computing, 13(4):1730-1 1742.
Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2016a. Effective Istms for target-dependent senti- ment classification. COLING, pages 3298-3307.
Duyu Tang, Bing Qin, and Ting Liu. 2016b. Aspect level sentiment classification with deep memory net- work. In EMNLP, pages 214-224
Sapna Negi and Paul Buitelaar. 2014. Insight galway: Syntactic and lexical features for aspect based senti- ment analysis. In SemEval, pages 346-30 350.
Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning re- sults. NewrIPS, 30.
Minh Hieu Phan and Philip O Ogunhona. 2020. Mod- elling context and syntactical features for aspect- based sentiment analysis. In ACL, pages 3211-3220.
Maria Pontiki, Dimitrios Galanis, Harris Papageorgiou, Ion Androutsopoulos, and et al. 2016. Semeval-2016 task 5: Aspect based sentiment analysis. In SemEval, pages 19-30.
Yuanhe Tian, Guimin Chen, and Yan Song. 2021. Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble. In NAACL, pages 2910-2922.

---

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Iliia Polosukhin. 2017. Attention is all you need. NewrPPS, 30.
have negative effects if they exist and are involved in training. In this work, we only randomly selected nearly fifty explanations for hand-checking. We find that the quality of the generated explanations is accurate and comprehensive (see examples in Table 6). In addition, We observe that these expla- nations are richly expressive (e.g., "not reasonable or affordable").
Bing Wang, Liang Ding. Qihuang Zhong, Ximing Li, and Dacheng Tao. 2022a. A contrastive cross- channel data augmentation framework for aspect- based sentiment analysis. In COLING, pages 6691- 6704.
Kai Wang, Weizhou Shen, Yunyi Yang. Xiaojun Quan, and Rui Wang. 2020. Relational graph attention net- work for aspect-based sentiment analysis. In ACL, pages 3229-3238.
Effect of Different Prompt Templates on Perfor- mance. this work, the prompt template is de- signed based on two points: (1) including a task de- scription (i.e., "explain the sentiment expressed by aspect term"), which is used to trigger the LLM's s ability to understand the task; (2) including the output limitation (i.e., "Limit forty words. Copy- ing adjectives from the original sentence is not allowed.") "), which is used as an output guide and prevents LLM from rephrasing the sentiment ex- pression. Here, to explore the effect of different prompt templates on performance, we try other well-designed templates and perform comparison experiments. These prompt templates are described in detail below:
Tianlu Wang. Rohit Sridhar, Diyi Yang. and Xuezhi Wang. 2022b. Identifying and mitigating spurious correlations for improving robustness in nlp models. In Findings of NAACL, pages 1719-1729.
Zhao Wang and Aron Culotta. 2020. Identifying spu- rious correlations for robust text classification. In Findings of EMNLP, pages 134--440.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H Chi, Quoe V Le, Denny Zhou, al. 2022. Chain-of-thouge prompting elicits rea- soning in large language models. In NeurlPS.
Jason Wei and Kai Zou. 2019. Eda: Easy data augmen- tation techniques for boosting performance text classification tasks. In EMNLP. pages 5382-6388-
Prompt 1 (used in this paper): In the follow- ing sentence X, explain why the sentiment expressed by aspect term a is y. Limit forty words. Copying adjectives from the original sentence not allowed.
Xiaoyu Xing, Zhijing Jin, Di Jin, Bingning Wang, Qi Zhang, and Xuan-Jing Huang. 2020. Tasty burg- soggy fries: Probing aspect robustness in aspect- based sentiment analysis. EMNLP, pages 3594- 3605.
Prompt In the following sentence X, ex- plain why the sentiment expressed by aspect term a is y.
Hu Xu, Bing Liu, Lei Shu, and S Yu Philip. 2019. Bert post-training for review reading comprehension and aspect-based sentiment analysis. In NAACL, pages 2324-2335.
Prompt 3: In the following sentence X, the sentiment expressed by aspect term a is y. Based on this prompt, explain its reasoning. Limit to forty words. Copying adjectives from the original sentence is not allowed.
Wei Xue and Tao Li. 2018. Aspect based sentiment analysis with gated convolutional networks. In ACI. pages
Hang Yan, Junqi Dai, Tuo Ji. Xipeng Qiu, and Zheng Zhang. 2021. A unified generative framework for aspect-based sentiment analysis. In ACL, pages 2416- 2429.
Here, for simplicity, choose BERT as the baseline model and 3ERT+augmeetation as our method. Table presents the experimental results. From this table, we can find that although the tem- plate used in this paper yields the best results, there is not much difference between the scores of the different templates (see Prompt vs. Prompt 3). Morcover, we observe that imposing an output con- straint in the template favors the performance (see Prompt I vs. Prompt 2).
Zheng Zhang, Zili Zhou, and Yanna Wang. 2022. Ssegen: Syntactic and semantic enhanced graph con- volutional network for aspect- based sentiment analy- sis. In NACCL, pages 4916-4925.
7 Appendix
Quality of the Automatically-Generaai Expla- nations. Large language models may generate less accurate or even irrelevant explanations for the sentiment of the aspect. Such explanations may
Error Analysis. An error analysis can provide readers with a deeper understanding of whether our

---

Dataset
Sample
Aspect
Sentiment
Explanation
The machine is slow boot up and boot up
negative
The phrase "boot up" implics that the chine is taking a long time start, which negative experience.
Lap14
sionally crashes completely.
Even though its good seafood, the prices prices are tno high.
negative
The prices are too high, which implies that they are not reasonable or affordable.
Rest14
USC cold smoker and smoked the avocado order to make the dish more complex.
dish
negative
The dish negative because avocado was snoked, which could make the dish taste unpleasant or unappetizing.
MAMS
Table 6: Examples of generating explanations. We can see that the explanation elucidates the reason (italicized content) the sentiment the aspect, grounded contextual semantics.
Sample
BERT
BERT+augmennttion
The [sta f flegutiee should a more friendly. Ineeeed a laptop with hig sttoragejeuutrnn a nice screen and fast can Photoshop without any problem.
positive(X) positive(X)
negative(v))
positive(XO)
Table 7: A simple error analysis. Aspect terms marked in parentheses, and the subscript indicates the corre- sponding sentiment polarity.
Datasets
Lap14
Rest14
Models
Acc.
F1
Acc.
Fl
BERT
75.80
71.67
82,59
74.10
BERT + augmentation
Prompt
77.74
73.11
84.29
76.07
+ Prompt
77.46
72.60
83.78
75.60
+ Prompt
77.50
72.94
84.08
75.72
Table 8:
Performance comparison of different prompts.
#eaT++augmenttiion using
The best scores are in bold.
methods have successfully reduced errors arising from statistical spurious correlations. Thus, we present simple error analysis in Table 7. Taking example, BERT makes
the first sample incorrect prediction possibly because of focusing on the word "friendly" only. We suspect this is because, the training samples which contain the word "friendly", 94.6% of aspect sentiments are "positive", i.e., statistically spurious correlation. Moreover, we find that the proposed method also makes few wrong predictions, especially when the true label is neutral, shown in the second sample. The potential reason may be that when the label neutral, the language model generates explanations with slight sentiments due to its own bias, which would mislead the model.


Reviews:
[{"review_id": "ifl1fO0AKc", "reviewer": "Reviewer_NFXw", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **`Response to Reviewer NFXw`**\n\nThanks very much for your valuable suggestions and comments. We sincerely appreciate your time reading the paper, and our point-to-point responses to your comments are below.\n\n**[Comment 1]** \n*Compared to other data augmentation baselines, the improvement achieved is relatively limited, considering that the framework could be resource-wise demanding since it requires loads of language model generation and training.*\n\n**[Response1]**\nThanks very much for your insightful comment! In this paper, we leverage explanations from LLMs to reduce spurious correlations in ABSA. Compared to other augmentation instances in which modified strategies are applied to the input text, these explanations are more readable. Also, they are more effective because they promote understanding of the contextual semantics when reasoning about the sentiments of the aspects rather than focusing only on certain words. Based on the experiments in Table 3, we can see that most of our scores are significantly better than those of other data augmentation baselines at a significance level of $p$ < 0.05.\n\nIn addition, another advantage of the proposed methods over other data augmentation baselines is the ability to improve the model's robustness. To test this point, we conduct experiments on the Aspect Robustness Test Set (Xing et al., 2020) and report results in the table below. We can find that the proposed methods (i.e., +augmentation and +distillation) yield a remarkable improvement compared to two popular augmentation baselines (i.e., +ADD and +DELETE). \n\nFinally, in order to clearly understand resource-wise demands, we would like to clarify that the proposed methods only load the language model for inference without involving training.\n\n\n|               |          **Lap14**         |  |  **Rest14**      ||\n|:-------------|:-----------:|:-----------:|:-----------:|:-----------:|\n|               | **Acc.**  | **F1**    | **Acc.**  | **F1**    |\n| BERT          |   60.58   |   56.41   |   60.74   |   52.02   |\n|          +ADD |   62.39   |   58.42   |   68.56   |   59.40   |\n|       +DELETE |   63.18   |   58.06   |   67.79   |   58.54   |\n| +augmentation | **66.82** | **62.28** |   74.13   | **66.00** |\n| +distillation |   66.09   |   61.12   | **74.22** |   65.47   |\n\n---\n\n\n**[Comment 2]** \n*Relying solely on unexamined and potentially biased language model generations requires more caution since language models cannot always provide accurate and comprehensive explanations. I would suggest conduct a closer investigation on LLM generated explanations and subsequently apply necessary quality control measures.*\n\n**[Response]**\nThank you for this constructive suggestion. Indeed, language models may generate less accurate or even irrelevant explanations for the sentiment of the aspect. Such explanations may have negative effects if they exist and are involved in training. In this work, we only randomly selected nearly fifty explanations for hand-checking. We find that the quality of the generated explanations is accurate and comprehensive (see examples in Table 7 in the Appendix). In response to your suggestion, we will purposefully design a module in the next version to determine the accuracy of the explanations generated by LLM and then take some control measures based on the accuracy.\n\n---\n\n\n**[Comment 3]**\n*Concerning reproducibility, the snapshot version and parameters of the LLMs used (GPT3.5 and MOSS) are not unspecified.*\n\n**[Response]**\nVery sorry for the lack of detail. The snapshot version and parameters of the GPT3.5 are [text-davinci-003](https://platform.openai.com/docs/models/gpt-3-5) and 175B, respectively. The snapshot version and parameters of the MOSS are [MOSS-moon-003-sft](https://github.com/OpenLMLab/MOSS) and 16B, respectively.\n\n---\n\n\nWe sincerely thank you for your positive feedback on our paper! If you have any questions or suggestions, please let us know!\n\nBest,\n\nAuthors", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The framework the study proposed achieved a general improvement on the ABSA task against multiple benchmark datasets.\n\nThe study includes solid experiments comparing various approaches on the specific task.", "reasons_to_reject": "Compared to other data augmentation baselines, the improvement achieved is relatively limited, considering that the framework could be resource-wise demanding since it requires loads of language model generation and training.\n\nRelying solely on unexamined and potentially biased language model generations require more caution, since language models could not always provide accurate and comprehensive explanations. I would suggest conduct a closer investigation on LLM generated explanations and subsequently apply necessary quality control measures.\n\nConcerning reproducibility, the snapshot version and parameters of the LLMs used (GPT3.5 and MOSS) are not unspecified.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper presents a 2-step framework that leverages explanations generated by large language models to augment existing aspect-based sentiment analysis (ABSA) dataset and so forth improve the performance on such task. The framework contains a prompt-based explanation generation and two simple yet effective explanation utilization strategies.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CL7YLxGBp0", "reviewer": "Reviewer_vdgb", "summary": "", "questions": "- Does the method mitigate all the spurious correlations?\n- Are there any new spurious correlations?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **`Response to Reviewer vdgb`**\n\nThanks very much for your valuable suggestions and comments. We sincerely appreciate your time in reading the paper, and our point-to-point responses to your comments are below.\n\n**[Comment 1]**\n*The lack of an error analysis to corroborate the claim.*\n\n**[Response]**\nThank you for the comment to improve our manuscript. An error analysis can provide readers with a deeper understanding of whether our methods have successfully reduced errors arising from statistical spurious correlations. In the next version, we will supplement the error analysis to better understand the claim of this paper. \n\nHere, in response to your comment, we present a simple error analysis in the table below. In Table, aspect terms are marked in parentheses, and the subscript indicates the corresponding sentiment polarity. Taking the first sample as an example, BERT makes an incorrect prediction possibly because of focusing on the word *\"friendly\"* only. We suspect this is because in the training samples which contain the word *\"friendly\"*, 94.6% of aspect sentiments are *\"positive\"*, i.e., statistically spurious correlation. In contrast, the proposed method can understand the contextual semantics rather than focusing on certain words when modeling the sentiment feature of the aspect. Moreover, we find that the proposed method also makes a few wrong predictions, especially when the true label is neutral, as shown in the second sample. The potential reason may be that when the label is neutral, the language model generates explanations with slight sentiments due to its own bias, which would mislead the model. \n\n|                                                  **Sample**                                                  | **BERT** | **BERT+augmentation** |\n|:------------------------------------------------------------------------------------------------------------|:----------:|:-----------------------:|\n| The $[staff]_{negative}$ should be a bit more friendly.                                                           | positive |        negative       |\n| I needed a laptop with big $[storage]_{neutral}$,  a nice screen and fast so I can Photoshop without any problem. | positive |        positive       |\n\n---\n\n**[Comment 2]**\n*Does the method mitigate all the spurious correlations?*\n\n**[Response]**\nThank you very much for the question. We regret to answer that mitigating all spurious correlations is absolute and difficult as well as somewhat impractical. In this paper, we aim to reduce these spurious correlations as significantly as possible and improve the model's generalization capabilities, as shown by various experiments.\n\n\n---\n\n**[Comment 3]**\n*Are there any new spurious correlations?*\n\n**[Response]**\nThank you for this constructive question. Indeed, detecting whether a few new spurious correlations are introduced is required after applying LLM to generate explanations. In this work, we only manually examine the quality of the random sampling explanations (see examples in Table 7). We find that these explanations are richly expressive (e.g., *\"not reasonable or affordable\"*). Therefore, we consider that they do not introduce new spurious correlations. Nevertheless, to answer this question accurately, we will subsequently find a way to do a quantitative analysis in the next version.\n\n---\n\nWe sincerely thank you for your positive feedback on our paper! If you have any questions or suggestions, please let us know!\n\nBest,\n\nAuthors", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The novelty of using LLM for augmenting data for ABSA.\n- The strong results of the two training approaches desinged.", "reasons_to_reject": "- The lack of an error analysis to corroborate the claim.", "questions_to_authors": "- Does the method mitigate all the spurious correlations?\n- Are there any new spurious correlations?", "paper_topic_and_main_contributions": "The paper presents a novel data augmentation strategy for ABSA (targeted sentiment classification) based on the use of large language models (LLM).\n\nI like the motivation of augmenting training data for ABSA for mitigating the spurious statistical correlation of the presence of some words with a specific sentiment label. Likewise, I see that the evaluation and analysis of the method is right. However, I miss an error analysis in order to see if the spurious correlations have disappeared.\n\nTo sum up, I think that it is a good paper with the lack of an error analysis.\n\n", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ROPTeV7gV5", "reviewer": "Reviewer_BR1K", "summary": "", "questions": "- Q1: One of the assumptions in this paper is that spurious correlations are an actual problem. E.g., Sec. 3.3 states this as a potential problem. Did you run any analysis how severe of an issue this actually is? How often does it occur for a given model on a given task? Which models are more/less prone to this?\n- Q2: L215: The definition assumes there's only one aspect per sentence. Are you considering sentences with multiple aspects?\n- Q3: Sec. 3.3: Prompt-tuning is a difficult task. How did you come up with this template? Did you try others? Consider adding more information into the appendix.\n- Q4: Sec 3: Did you run any analysis to probe for the quality of the automatically-generated explanations? I'd be curious if the explanations are accurate.\n- Q5: Sec. 3.4 augmentation-based method: This approach supplies more training data to the classifier (according to Fig 2), which gives it an advantage compared to the baseline model. Have you compared the performance when keeping the number of training examples equal (e.g., by using 50% original, 50% explanations)? Without this we can't say for sure if improvements come from explanations or simply more training data.\n- Q6: Eq. 2: I need some help understanding the Equations and the explaining text in L324-326. It could mean that the goal is to minimize the KL divergence between the predictions of the original example and the explanation for this example. In this case $g$ would be the student. Alternatively, it could mean that the goal is to minimize the KL divergence from the teacher's predictions and the student's predictions (which is the typical knowledge distillation process). In that case Eq. 2 would need to distinguish teacher and student and use $X'$ instead of $X$ or $X_hat$, e.g.,$KL(g_{teacher}(X'_i), g_{student}(X'_i)$. Similarly for Eq. 3. I may misinterpret something so I'd be grateful for any clarification.\n- Q7: L412: Since we don't know what percentage of the data suffers from spurious correlation this claim is unjustified. The numbers simply show that more training data (augmentation) and distillation improves on the datasets. That in itself is nice, but there's (not necessarily) a relation to spurious correlations. On top of that we can't tell from the experiments if the given explanations are causing improvements or if its simply about adding more data in general.\n- Q8: L420: Again, this statement is too strong. It may be a hypothesis, but not a fact that's shown by the experiments. It would help the reader to have a significance test to judge the difference in performance. E.g., Bert+PT vs GPT2 on Lag14 is < .5%, which is only 12 examples on this dataset.\n- Q9: Tab 2: Does \"+ distillation mean\": \"model + distillation\" or \"model + augmentation + distillation\"?\n- Q10: Sec 5, L456: Did you replace training examples with their augmented versions or add them to the existing set?\n- Q11: Sec 5, L473: It would be useful to add 1 or 2 sentences how the adversarial setting works.\n- Q12: L513: The interpretation here depends on my earlier question Q5. Does the augmentation model have twice the amount of data? If so it's not surprising that the model performs better and it's not clear if the improvement comes purely from more data or from the explanations themselves.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **`Response to Reviewer BR1K`**\n\nThanks very much for your valuable suggestions and comments. We sincerely appreciate your time in reading the paper, and our point-to-point responses to your comments are below.\n\n**[Comment 1]**\n*My main concern with the paper is that it's focused around fixing spurious correlations but there's no analysis how much of a problem that actually is in any of the used datasets. Many of the experimental results are used as claims to show how spurious correlations are prevented but there's no proof that this is actually the case. Yes, performance of the models improve with the proposed methods (which in itself is a nice finding). However, we can't say that improvement are caused by less spurious correlations in the model; making contribution 1 (L141) invalid. In my view the paper needs more support for the claims or a rewrite away from spurious correlations.*\n\n**[Response]**\nThanks very much for your insightful comments and suggestions! In this work, spurious correlation (Zhang et al., 2022; Wang et al., 2022b) refers to the dependence of the model on certain words in the input text without a deeper understanding of the contextual semantics, which has a know-it-when-you-see-it character. **To the best of our knowledge, there are no relevant studies that analyze how much of the spurious correlation actually is in the used datasets. Existing related work only illustrates the presence of spurious correlations in datasets in the form of case studies.** Here, to answer your concerns and provide more support, we conduct a simple probe experiment on the Aspect Robustness Test Set (ARTS) (Xing et al., 2020). ARTS enriches the initial test sets from Lap14 and Rest14 by employing adversarial strategies (see some examples in Table below). \n\n|                     |                     **Sample**                    | **Aspect** | **Label** |\n|:---------------------|:-------------------------------------------------|:------------:|:-----------:|\n|   original sample   | Thanks for the fast shipment and great price.     |  shipment  |  positive |\n| adversarial sample | Thanks for the slow shipment and great price.     |  shipment  |  negative |\n| adversarial sample       | Thanks for the fast shipment and not great price. |  shipment  |  positive |\n\n\nHere, *if a model predicts the same sentiment labels for an original sample as well as its adversarial samples (their true labels are different), we will assume that there is a spurious correlation between certain contextual words (such as *\"Thanks\"*) and the predicted label (such as *\"positive\"*) for this original sample.* In other words, the predicted label does not change with the contextual semantics because the model only focuses on certain words.-- Based on this assumption, we count the percentage of original samples in the test set that contain spurious correlations (the lower the better). *It is worth reminding that this percentage is not the actual percentage of spurious correlations in the dataset, which is only an estimate under this assumption.* According to this table, we can see that the spurious correlations exist in the Lap14 and Rest14 datasets. Moreover, we can observe that the proposed methods reduce the percentage of original samples containing spurious correlations. This may suggest that the generated explanations can alleviate the spurious correlations problem in the ABSA task.\n\n|               | **Lap14** | **Rest14** |\n|---------------|:---------:|------------|\n| BERT          |   14.8%   |    14.5%   |\n| +augmentation |   12.7%   |    11.6%   |\n| +distillation |   13.0%   |    11.4%   |\n| AEN           |   16.3%   |    16.4%   |\n| +augmentation |   15.2%   |    14.8%   |\n| +distillation |   15.3%   |    15.0%   |\n| RGAT          |   12.4%   |    12.5%   |\n| +augmentation |   11.1%   |    11.6%   |\n| +distillation |   10.8%   |    11.0%   |\n\n>[1] Zhang, Michael, et al. \"Correct-N-Contrast: a Contrastive Approach for Improving Robustness to Spurious Correlations.\" International Conference on Machine Learning. PMLR, 2022.\n\n>[2] Wang, Tianlu, et al. \"Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models.\" Findings of the NAACL. 2022.\n\n>[3] Xing, Xiaoyu, et al. \"Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis.\" Proceedings of EMNLP. 2020.\n\n---\n\n**[Comment 2]**\n*Some details are missing to fully judge the quality of the improvements and a simple baseline without adding additional training data is missing (see details below).*\n\n**[Response]**\nThank you for this constructive comment. Since Question 5 is the detailed version of this comment, we will answer your concern in Answer 5.\n\n---\n---\n---\n\n**[Q1]:**  *One of the assumptions in this paper is that spurious correlations are an actual problem. E.g., Sec. 3.3 states this as a potential problem. Did you run any analysis how severe of an issue this actually is? How often does it occur for a given model on a given task? Which models are more/less prone to this?*\n\n**[A1]:** Thank you for these questions to improve our manuscript. We answer these questions with the table in the response to Comment 1 and Table 4.  \n\nFor the first question, spurious correlations seriously hurt performance (see Table 4). This is because although they are a relatively small percentage (see the table above), they can prevent the model from achieving satisfactory results on the ARTS dataset. These results just amplify the harmful nature of this issue.\n\nFor the second question, for a given model on a given task, how often it occurs depends on the training dataset. Suppose that if aspect sentiments are all *\"positive\"* in the training samples in which the word *\"fast\"* is contained, for a test sample containing *\"fast\"*, the trained model will predict *\"positive\"* with high probability regardless of the contextual semantics.\n\nFor the third question, we can observe based on the table above that the attention\\-based models (or the graph\\-based models) may be more (or less) prone to this issue. We conjecture that the attention\\-based models are more prone to concentrate on certain sentiment\\-related words rather than understanding the entire contextual semantics. This may easily lead to statistically spurious correlations. In contrast, the graph\\-based models can better understand the whole contextual semantics through the propagation of information between nodes.\n\n---\n\n**[Q2]:**  *L215: The definition assumes there's only one aspect per sentence. Are you considering sentences with multiple aspects?*\n\n**[A2]:** Sorry for the unclear expression. For sentences with multiple aspects, we treat other non-targeted aspects as normal context tokens when focusing on the target aspect. In other words, a sentence will be processed multiple times, which is equal to the number of aspects it contains.\n\nHere, taking *\"Great food but the service was dreadful!\"* as an example for better illustration, we process it twice. For the first process, the model receives the aspect and sentiment as *\"food\"* and *\"positive\"*. For the second process, the model takes *\"service\"* and *\"negative\"* as the aspect and sentiment.\n\n---\n\n\n**[Q3]:**  *Sec. 3.3: Prompt-tuning is a difficult task. How did you come up with this template? Did you try others? Consider adding more information into the appendix.*\n\n**[A3]:** Thanks for your advice. This prompt template is designed based on two points: (1) including a task description (i.e., *\"explain the sentiment expressed by aspect term\"*), which is used to trigger the LLM's ability to understand the task; (2) including the output limitation (i.e., *\"Limit to forty words. Copying adjectives from the original sentence is not allowed.\"*), which is used as an output guide and prevents LLM from rephrasing the sentiment expression.\n\nTo further answer your concern, we try other well-designed templates here and perform comparison experiments. These templates are described in detail below:\n- **Prompt 1** (used in this paper): In the following sentence $X$, explain why the sentiment expressed by aspect term $a$ is $y$. Limit to forty words. Copying adjectives from the original sentence is not allowed.\n- **Prompt 2**: In the following sentence $X$, explain why the sentiment expressed by aspect term $a$ is $y$. \n- **Prompt 3**: In the following sentence $X$, the sentiment expressed by aspect term $a$ is $y$. Based on this prompt, explain its reasoning. Limit to forty words. Copying adjectives from the original sentence is not allowed.\n\nHere, for simplicity, we choose BERT as the baseline model and BERT+augmentation as our method. The table below presents the experimental results. From this table, we can find that although the template used in this paper yields the best results, there is not much difference between the scores of the different templates (see Prompt1 vs. Prompt3). Moreover, we can observe that imposing an output constraint in the template favors the performance (see Prompt1 vs. Prompt2).\n\n|               |         | **Lap14** |        | **Rest14** |        |\n|---------------|---------|:---------:|:------:|:----------:|:------:|\n|               |         |  **Acc.** | **F1** |  **Acc.**  | **F1** |\n| BERT          |         |   75.80   |  71.67 |    82.59   |  74.10 |\n| +augmentation | +Prompt1 |   **77.74**   |  **73.11** |    **84.29**   |  **76.07** |\n| +augmentation | +Prompt2 |   77.46   |  72.60 |    83.78   |  75.60 |\n| +augmentation | +Prompt3 |   77.50   |  72.94 |    84.08   |  75.72 |\n\n\n---\n\n**[Q4]:**  *Sec 3: Did you run any analysis to probe for the quality of the automatically\\-generated explanations? I'd be curious if the explanations are accurate.*\n\n**[A4]:** Many thanks for this interesting question. In this work, we only randomly selected nearly fifty explanations for hand-checking. We find that the quality of the automatically generated explanations is satisfactory and accurate (see some examples in Table 7 in the Appendix). However, we did not quantitatively evaluate the quality of the explanations, which is an oversight on our part. In the revision version, we will try to find an approach to quantitatively probe the accuracy of the automatically generated explanations.\n\n---\n\n**[Q5]:**  *Sec. 3.4 Augmentation\\-based method: This approach supplies more training data to the classifier (according to Fig 2), which gives it an advantage compared to the baseline model. Have you compared the performance when keeping the number of training examples equal (e.g., by using 50% original, 50% explanations)? Without this we can't say for sure if improvements come from explanations or simply more training data.*\n\n**[A5]:** We are very grateful for this insightful question. To address your concern, we perform comparative experiments under different settings in which the proportions of original examples and explanations change but the totals remain consistent. The table below gives detailed experimental results. By comparing results when keeping the number of training examples equal, we can observe that the explanations can improve performance. However, we find that the magnitude of performance improvement does not increase as the proportions of explanations increase. We conjecture that the explanations have limited performance improvement under keeping the total number of training samples equal. \n\nMoreover, we can observe from Table 3 that even when doubling the number of training examples (e.g., by using 100% original, 100% explanations or augmentations), the proposed method obtains better results when compared to other data augmentation models. This suggests that the explanations used in this paper are indeed effective and can contribute to performance improvement.\n\n|          |              | **Lap14** |           | **Rest14** |           |\n|----------|--------------|:---------:|:---------:|:----------:|:---------:|\n| original | explanations |  **Acc.** |   **F1**  |  **Acc.**  |   **F1**  |\n|   100%   |      0%      |   75.80   |   71.67   |    82.59   |   74.10   |\n|    70%   |      30%     |   75.93   |   71.89   |    82.71   |   74.45   |\n|    50%   |      50%     |   76.45   |   72.53   |    83.60   |   75.03   |\n|    30%   |      70%     |   76.23   |   72.36   |    83.43   |   75.11   |\n|   100%   |     100%     | **77.74** | **73.11** |  **84.29** | **76.07** |\n\n---\n\n**[Q6]:**  *Eq. 2: I need some help understanding the Equations and the explaining text in L324\\-326. It could mean that the goal is to minimize the KL divergence between the predictions of the original example and the explanation for this example. In this case $g$ would be the student. Alternatively, it could mean that the goal is to minimize the KL divergence from the teacher's predictions and the student's predictions (which is the typical knowledge distillation process). In that case Eq. 2 would need to distinguish teacher and student and use $X'$ instead of $X$ or $\\hat{X}$, e.g.,$KL(g_{teacher}(X'_i),g\\_{student}(X'_i))$. Similarly for Eq. 3. I may misinterpret something so I'd be grateful for any clarification.*\n\n\n**[A6]:** Thanks for your meticulous review and for raising questions about the equations and associated explaining text in lines 324-326. You are correct in your understanding. Our goal is indeed to minimize the KL divergence between the predictions of the original sample and the explanation for this sample. The equations we provided for this purpose aim to mathematically describe this process. In light of your feedback, we will revise these two equations and make the necessary text modifications to differentiate clearly between the teacher and student models.\n\nThank you again for pointing out these ambiguities!\n\n\n\n---\n**[Q7]:**  *L412: Since we don't know what percentage of the data suffers from spurious correlation this claim is unjustified. The numbers simply show that more training data (augmentation) and distillation improves on the datasets. That in itself is nice, but there's (not necessarily) a relation to spurious correlations. On top of that we can't tell from the experiments if the given explanations are causing improvements or if its simply about adding more data in general.*\n\n**[A7]:** Thanks very much for your insightful comments! Here, we address your concerns from two perspectives: (1) According to the table in the response to Comment 1, the percentage of the data that suffers from spurious correlation decreases after applying the explanations. This shows that the proposed explanations can effectively mitigate spurious correlations; (2) According to the Answer to the corresponding Question5, we can know that the explanations can bring performance improvement when keeping the number of training examples equal. In the revision, we will supplement a detailed description so that readers can better understand our work.\n\n\n---\n\n**[Q8]:**  *L420: Again, this statement is too strong. It may be a hypothesis, but not a fact that's shown by the experiments. It would help the reader to have a significance test to judge the difference in performance.*\n\n**[A8]:** Thanks for your useful advice. In the next version, we will re-examine the paper and modify inappropriate statements. Here, in response to your suggestion, we add a significance test in the table below to judge the difference in performance described by line 420. The symbol \\* denotes that the score is significantly better than that of GPT2 at a significance level of $p < 0.05$.    We can see that although the number of parameters in GPT2 is slightly larger than that in BERT+PT, BERT+PT is overall better than GPT2. The potential reason may be that understanding the context sufficiently related to the given aspect is difficult when using a text generation paradigm to solve ABSA.\n\n|         | **Lap14** |        | **Rest14** |        | **Rest15** |        | **Rest16** |        | **MAMS** |        |\n|:-------:|:---------:|:------:|:----------:|:------:|:----------:|:------:|:----------:|:------:|:--------:|:------:|\n|         |  **Acc.** | **F1** |  **Acc.**  | **F1** |  **Acc.**  | **F1** |  **Acc.**  | **F1** | **Acc.** | **F1** |\n| BERT+PT |   77.59   |  $73.14^*$ |    84.16   |  76.62 |    $82.83^*$   |  $65.81^*$ |    $91.73^*$   |  $73.93^*$ |   83.48  |  $83.10^*$ |\n|   GPT2  |   76.04   |  72.56 |    84.11   |  77.02 |    81.06   |  64.90 |    89.96   |  71.90 |   82.86  |  81.61 |\n\n---\n**[Q9]:**  *Tab 2: Does \"+ distillation mean\": \"model + distillation\" or \"model + augmentation + distillation\"?*\n\n**[A9]:** Sorry for the unclear description. In Table 2, *\"+ distillation\"* refers to *\"model + distillation\"*.\n\n---\n\n**[Q10]:** *Sec 5, L456: Did you replace training examples with their augmented versions or add them to the existing set?*\n\n**[A10]:** We apologize for not going into detail. In line 456, we add these augmented versions directly to the existing set. The size of the training set will be doubled after this operation.\n\n---\n\n**[Q11]:** *Sec 5, L473: It would be useful to add 1 or 2 sentences how the adversarial setting works.*\n\n**[A11]:** Thanks for your useful suggestions. In line 473, to show that our methods can reduce spurious correlations (i.e., robustly understand the contextual semantics in which the aspect sentiment resides rather than focusing on certain words), we conduct experiments on the Aspect Robustness Test Set (ARTS). This dataset simulates some adversarial scenarios. Specifically, ARTS extends the original test samples of Lap14 and Rest14 using three adversarial settings: (1) reversing the original sentiment of the targeted aspect; (2) reversing the sentiment of the non\\-targeted aspects, and (3) generating more non-targeted aspects with opposite sentiment polarities from the targeted aspect. Thus, this dataset is able to test whether the model can robustly capture the aspect\\-relevant contextual semantics to distinguish the sentiment towards the targeted aspect from the input text.\n\n\n\n---\n\n**[Q12]:**  *L513: The interpretation here depends on my earlier question Q5. Does the augmentation model have twice the amount of data? If so it's not surprising that the model performs better and it's not clear if the improvement comes purely from more data or from the explanations themselves.*\n\n**[A12]:** We apologize for the unclear description. In line 513, the augmentation model has twice the amount of data. However, as described in Anaswer5, the explanations do bring performance improvement. Thus, we could roughly conclude that the performance improvement of line 513 comes not only from more data but also from the explanations themselves.\n\n---\n\n**`Typos Grammar Style and Presentation Improvements`**\n\nThanks for your detailed reading! In order to improve the quality of the presentation, we will fix the typos and bugs you pointed out in the revision following your suggestions.\n\n\n\n---\n\nWe sincerely thank you for your positive feedback on our paper! If you have any questions or suggestions, please let us know!\n\nBest,\n\nAuthors", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed methods are intuitive (e.g., how to get an explanation and how to use them) and the many experiments show deep analysis and an attempt to study the problem from many different perspectives (e.g., text length, alternative text augmentation methods, robustness and generalization experiments).", "reasons_to_reject": "- My main concern with the paper is that it's focussed around fixing spurious correlations but there's no analysis how much of a problem that actually is in any of the used datasets. Many of the experimental results are used as claims to show how spurious correlations are prevented but there's no proof that this is actually the case. Yes, performance of the models improve with the proposed methods (which in itself is a nice finding). However, we can't say that improvement are caused by less spurious correlations in the model; making contribution 1 (L141) invalid. In my view the paper needs more support for the claims or a rewrite away from spurious correlations.\n- Some details are missing to fully judge the quality of the improvements and a simple baseline without adding additional training data is missing (see details below).", "questions_to_authors": "- Q1: One of the assumptions in this paper is that spurious correlations are an actual problem. E.g., Sec. 3.3 states this as a potential problem. Did you run any analysis how severe of an issue this actually is? How often does it occur for a given model on a given task? Which models are more/less prone to this?\n- Q2: L215: The definition assumes there's only one aspect per sentence. Are you considering sentences with multiple aspects?\n- Q3: Sec. 3.3: Prompt-tuning is a difficult task. How did you come up with this template? Did you try others? Consider adding more information into the appendix.\n- Q4: Sec 3: Did you run any analysis to probe for the quality of the automatically-generated explanations? I'd be curious if the explanations are accurate.\n- Q5: Sec. 3.4 augmentation-based method: This approach supplies more training data to the classifier (according to Fig 2), which gives it an advantage compared to the baseline model. Have you compared the performance when keeping the number of training examples equal (e.g., by using 50% original, 50% explanations)? Without this we can't say for sure if improvements come from explanations or simply more training data.\n- Q6: Eq. 2: I need some help understanding the Equations and the explaining text in L324-326. It could mean that the goal is to minimize the KL divergence between the predictions of the original example and the explanation for this example. In this case $g$ would be the student. Alternatively, it could mean that the goal is to minimize the KL divergence from the teacher's predictions and the student's predictions (which is the typical knowledge distillation process). In that case Eq. 2 would need to distinguish teacher and student and use $X'$ instead of $X$ or $X_hat$, e.g.,$KL(g_{teacher}(X'_i), g_{student}(X'_i)$. Similarly for Eq. 3. I may misinterpret something so I'd be grateful for any clarification.\n- Q7: L412: Since we don't know what percentage of the data suffers from spurious correlation this claim is unjustified. The numbers simply show that more training data (augmentation) and distillation improves on the datasets. That in itself is nice, but there's (not necessarily) a relation to spurious correlations. On top of that we can't tell from the experiments if the given explanations are causing improvements or if its simply about adding more data in general.\n- Q8: L420: Again, this statement is too strong. It may be a hypothesis, but not a fact that's shown by the experiments. It would help the reader to have a significance test to judge the difference in performance. E.g., Bert+PT vs GPT2 on Lag14 is < .5%, which is only 12 examples on this dataset.\n- Q9: Tab 2: Does \"+ distillation mean\": \"model + distillation\" or \"model + augmentation + distillation\"?\n- Q10: Sec 5, L456: Did you replace training examples with their augmented versions or add them to the existing set?\n- Q11: Sec 5, L473: It would be useful to add 1 or 2 sentences how the adversarial setting works.\n- Q12: L513: The interpretation here depends on my earlier question Q5. Does the augmentation model have twice the amount of data? If so it's not surprising that the model performs better and it's not clear if the improvement comes purely from more data or from the explanations themselves.", "paper_topic_and_main_contributions": "The paper proposes a 2 step process to address spurious correlations in aspect-based sentiment analysis. Step 1: generate an explanation for a (sentence, aspect, polarity) triple using an LLM. Step 2: either augment the training data of a classifier using these explanations or using the explanations in a distillation setup.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Fig 2, step 1\n\t- The shadow under the text makes it difficult to read. I suggest deactivating it.\n\t- The turquoise text in the right-most box is pretty much unreadable. Please also consider using colors that are strong in contrast to also support readers that are visually impaired.\n- L256: \"unexplored\" -> \"explored\"\n- L317: nit: the footnote should go behind the period.\n- Be careful with shrinking vertical spaces, e.g., Eq. 3 blends into the text below it.\n- Tab 1: nit: align the numbers to the right\n- Sec. 3.4, L304: I suggest to add a reference to Sec 5 to support this claim.\n- L335: Careful with $\\theta^T$, as this is standard mathematical notation for transposition. Maybe use $\\theta$ for teacher and $\\phi$ for student weights."}}]

Author rebuttal:

Meta-review: This paper introduces a 2-step framework using large language models (LLMs) to augment aspect-based sentiment analysis (ABSA) datasets, aiming to improve performance. The framework involves prompt-based explanation generation and two explanation utilization strategies. 

Reasons to Accept:
- The proposed framework demonstrates a significant overall improvement in aspect-based sentiment analysis (ABSA) across multiple benchmark datasets.
- The study features robust experiments that compare various approaches on the specific task, providing strong empirical evidence.
-The novelty of leveraging large language models (LLMs) for data augmentation in ABSA
-The study presents two effective training approaches, enhancing the overall quality of the proposed methods.
-The methods are intuitive, and the extensive experiments offer a deep analysis from various perspectives, including text length, alternative text augmentation methods, robustness, and generalization.

Reasons to Reject:
- The observed improvement achieved with the proposed framework is relatively limited when compared to other data augmentation baselines. Given the resource-intensive nature of the framework, this raises concerns about its cost-effectiveness.
- Relying solely on unexamined and potentially biased language model-generated explanations may not be a reliable approach, as language models may not consistently provide accurate or comprehensive explanations. A closer investigation of LLM-generated explanations and the application of quality control measures are needed.
- The paper lacks transparency regarding the snapshot version and parameters of the LLMs used (GPT-3.5 and MOSS), which affects reproducibility.
- The absence of an error analysis to validate the claim is a significant drawback.
Decision: Accept-Findings