CONTRASTE: Supervised Contrastive Pre-training With Aspect-based Prompts For Aspect Sentiment Triplet Extraction
Authors: Rajdeep Mukherjee
Nithish Kannen
Saurabh Kumar Pandey
Pawan Goyal
Venue: EMNLP 2023 Findings (2023)
Keywords: supervised contrastive learning
pretraining
t5
encoder-decoder
generative
aste
acos
aesc
tasd
absa
TL;DR: CONTRASTE: Supervised Contrastive Pre-training With Aspect-based Prompts For ASTE

Abstract: Existing works on Aspect Sentiment Triplet Extraction (ASTE) explicitly focus on developing more efficient fine-tuning techniques for the task. Instead, our motivation is to come up with a generic approach that can improve the downstream performances of multiple ABSA tasks simultaneously. Towards this, we present CONTRASTE, a novel pre-training strategy using CONTRastive learning to enhance the ASTE performance. While we primarily focus on ASTE, we also demonstrate the advantage of our proposed technique on other ABSA tasks such as ACOS, TASD, and AESC. Given a sentence and its associated (aspect, opinion, sentiment) triplets, first, we design aspect-based prompts with corresponding sentiments masked. We then (pre)train an encoder-decoder model by applying contrastive learning on the decoder-generated aspect-aware sentiment representations of the masked terms. For fine-tuning the model weights thus obtained, we then propose a novel multi-task approach where the base encoder-decoder model is combined with two complementary modules, a tagging-based Opinion Term Detector, and a regression-based Triplet Count Estimator. Exhaustive experiments on four benchmark datasets and a detailed ablation study establish the importance of each of our proposed components as we achieve new state-of-the-art ASTE results.

CONTRASTE:
Supervised Contrastive Pre-training With Aspect-based Prompts For Aspect Sentiment Triplet Extraction
Rajdeep Mukherjee
Nithish Kannen Saurabh Kumar Pandey Indian Institute Of Technology Kharagpur (ejjeepp1899iitteee ac. saurabh2000. ii tiiktggggaal...mmmmmmmm com, pawangêcse. iitkgp. ac. in
Pawan Goyal
ac. in, nithishhamme@@gail. com,
Abstract
2021; Chen al., 2021; Yan et al., 2021; Zhang et al., 2021a) la) have all developed newer, better, and often more complex fine-tuning techniques for the task, gradually enhancing performance on the benchmark datasets (Xu et al., 2020). Our moti- vation instead is to essentially improve the aspect- aware sentiment understanding of generative ar- chitecture that simultaneously benefit multi- ple ABSA tasks. Towards this direction, we pro- pose CONTRASTE, CONTRAstive learning- based efficient pre-training strategy that enhances the downstream performance of ASTE. Although we primarily focus on ASTE, we also demonstrate the advantage of our proposed approach in achiev- ing state-of-the-ae performances for other ABSA tasks such Aspect Category Opinion Sentiment (ACOS) quad prediction, Target Aspect Sentiment Detection (TASD). and Aspect Extraction and Sen- timent Classification (AESC).
Existing works on Aspect Sentiment Triplet Extraction (ASTE) explicitly focus on devel- oping more efficient fine-tuning techniques for the task. Instead, our motivation is to come up with a generic approach that can improve the downstream performances of multiple ABSA tasks simultaneously. Towards this, we present CONTRASTE, a novel pre-training strategy using CONTRastive learning to enhance the ASTE performance. While we primarily fo- ASTE, we also demonstrate the advan- tage of our proposed technique on other ABSA tasks such ACOS, TASD, and AESC. Given a sentence and its associated (aspect, opin- ion, sentiment) triplets, first, we design aspect- based prompts with corresponding sentiments masked. We then (pre)train encoder-decoder model by applying contrastive learning on the decoder-generated aspect-aware sentiment rep- resentations of the masked terms. For fine- tuning the model weights thus obtained, we then propose novel multi-task approach where the base encoder-decoder model is combined with two complementary modules, a tagging- based Opinion Term Detector, and regression- based Triplet Count Estimator. Exhaustive ex- periments on four benchmark datasets and de- tailed ablation study establish the importance of each our proposed components as we achieve new state-of-the-ar ASTE results.
Supervised Contrastive Learning (SCL) (Khosla et al., 2020) has been previously explored in ABSA (Li et al., 2021b; b; Liang et al., 2021; Ke al., 2021). SCL is performed on label (here sentiment) repre- sentations, whose distance in the embedding space, during training, is pulled together from the ones with the same sentiment orientation and pushed apart from those with different sentiment polarities. The existing works however differ from ours in their objectives and approaches. A detailed com- parison is presented in Section 5. Most importantly, the existing techniques apply SCL on sentence- level sentiment embeddings, whereas in ASTE (or ABSA in general), the sentiments are defined at an aspect level. To address this limitation, we intro- duce aspect-based prompts with [MASK] tokens. A sentence, for e.g., the food was delicious. is sent as input to the T5 (Raffel et al., 2019) encoder. The decoder is then provided with an aspect-based prompt, <aspect> food <sentiment> [MASK]. The decoded representation for the [MASK] token gives us the espect-aware representation of the cor-
1 Introduction
ASTE is the most interpretable Aspect-based Sen- timent Analysis (ABSA) task that, given a sen- tence, requires the extraction of opinion triplets, each consisting of an aspect term, the sentiment ex- pressed towards it, and the corresponding opinion term explaining the rationale behind the sentiment, shown in Table 2. Existing ASTE approaches, ranging from pipeline (Peng al., 2020a), multi- task (Zhang et al., 2020), tagging- (Xu et al., 2020; Wu et al., 2020), span-based (Xu et al., 2021; Feng et al., 2022), and generative (Mukherjee et al.,

---

Pre-training Prompts
Sentence
<aspect> aspect <sentiment> [MASK]
The food was good.
<aspect> food <sentiment> [MASK]
<aspect> sound <sentiment> [MASK]
Both sound as well as display quality are great.
<aspect> display quality <sentiment> [MASK]
<aspect> sushi <sentiment> [MASK]
While the sushi was tasty, the amhience sucked.
<aspect> ambience <sentiment> [MASK]
Table I: Few sentences along with aspect-based prompts derived from them to pre-train an enc-dec framework.
responding sentiment label. Aspect-level SCL is performed on these embeddings. Our proposed pre- training scheme is depicted in Fig 1(a). Few exam- ple sentences along with the aspect-based prompts derived from them to pre-train the framework are shown in Table 1. In Section 3.5, we compare the two pre-training approaches, that is performing SCL on sentence-level versus aspect-level senti- ment embeddings, and demonstrate that the latter (our proposal) leads better ASTE performance.
level sentiment embeddings for pre-training. as in existing works. Also, different from prior works, we do not use any additional data to perform contrastive pre-training of our model.
We propose more generic placeholder-basee tem- plates than the ones used in PARAPHRASE (re- fer to Table 2) to fine-tune T5 for ASTE. We further demonstrate that the template can be eas- ily customized for various ABSA tasks. Our final model, CONTRASTE-MTLI (Fig. 1(b)), pre-trained using SCL on aspect-level sentiment embeddings, and fine- ine-ttned for ASTE using a multi-task objective, achieves statc-of-the-art (SOTA) results on all four benchmark datasets. We also show that our proposed pre-training strat- egy helps in achieving SOTA results for other ABSA tasks such ACOS, TASD, and ASEC. Finally, we compare our results for all four tasks against ChatGPT (OpenAI, 2023b), and observe substantial gains in performance.
After pre-training completed, we need to fine- tune the trained model weights for the downstream ABSA tasks (we focus on ASTE here). Simi- lar to PARAPHRASE (Zhang et al., 2021a), the most competitive generative approach for ASTE and other ABSA tasks, train our base encoder- decoder framework, CONTRASTE-Base. to gen-
erate opinion triplets as templatized paraphrases, as shown in Fig. 1(b). Our template however dif-
fers. We argue that the template used in PARA- PHRASE, although more intuitive, does not gener- alize well across tasks, and often results in seman- tically meaningless target sequences, as reported in Table 2. Taking inspiration from Huguet Cabot and Navigli (2021), we therefore design a more generic placeholder- template that can be leveraged across ABSA tasks. Finally, we train CONTRASTE- Base with two auxiliary multitask objectives, Opin- ion Term Detection and Triplet Count Estimation, each of which benefits the ASTE performance. We call our final model CONTRASTE-MTL. be summarized as follows: We propose novel strategy to continually pre- train T5 for ABSA tasks (from its generic pre- trained checkpoint). More specifically, given a sentence, we derive (possibly multiple) aspect- based prompts with corresponding sentiments masked as shown in Table 1. The model is then pre-trained by applying supervised contrastive learning (SCL) on the decoder--gnneaaeed aspect- level sentiment representations of the masked tokens as shown in Fig (a). We show that such an approach leads to better downstream ASTE performance than performing SCL on sentence-
Section 2 presents the details of our pro- posed pre-training and fine-tuning methodologies. Section 3.4 demonstrates the advantage of our placeholder-based templates, over the ones used in PARAPHRASE, in achieving better ASTE perfor- mance. Section 3.5 compares our pre-training strat- egy with the one used in prior works. Here, we visu- ally demonstrate how pre-training on aspect-centric sentiment embeddings results in deriving more dis- cernible clusters of representations with different sentiment polarities. Our main results are reported in Section 3.6. Section 4 discusses our model ab- lations and demonstrates how pre-traue pre-training helps in improving the performances of other ABSA tasks, such as ACOS, TASD, and AESC.
Our contributions
Methodology
2.1 Supervised Contrastive Pre-training
We model ASTE a structured prediction task, and leverage the T5 (Raffel et al., 2019) encoder- decoder framework as the backbone of our pro- posed architecture. The empirical justification for this design choice primarily comes from prior

---

Oljcriie
Qeban Tarrs Detadar
caqpett oppiear pareiz nealaszall
ttittttf
Rood tabalzas anbiian posr
<аироcо andienct COnUIEENT
aweonnd!
cale
the Gak
+lps
(a) Supervised contrastive pre-training
(b) Proposed multi-task model for ASTE
Figure CONTRASTE: (a) Contrastive pre-training of the encoder-decoder framework using aspect-based prompts. (b) Fine-tuning the model for ASTE by optimizing joint-ojecctive generate template-based triplets.
CONTRASTE template:
PARAPHRASE template: ok bad because ASPECT OPINION
SENTENCE
<aspect> agper Kopi
ssentinent> sentiment [SSEP]
The food good. While the sushi was usily,
<aspect> t> lood capinion2 Esentinent> POS
because food is good
<espect> susli capiniono lasly isentinents> POS [SSEP]
Il great because sushi is lasty [SSEP] is bad because ambience is sucked
ambience sucked. very drrappoinedd with the chef.
<aspect> umbience copinion> sucked <sentinents> NEG
<aspect> chef copiniono very disappointed iirett> NEG
bad because chef very disappointed
Table 2: Few sentences along with their corresponding targets, to fine-tune the T5 encoder-decoder framework, generated using our proposed templates vs. the ones proposed by PARAPHRASE (Zhang et al., 2021a). The target sequences highlighted in red are not semantically meaningful: especially the last example where the customer should be disappointed, and not the chef, contrary to what is meant by the paraphrased sentence.
works. Earlier ASTE approaches like JET-BERT (Xu et al., 2020), GTS-BER: (Wu al., 2020), and Span-ASTE (Xu al., 2021) are tagging- based approaches built upon single encoder. Later works such as PASTE (Mukherjee et al., 2021), Unified-BART-BBAA (Yan et al., 2021), and PARAPHRASE (Zhang et al., 2021a) have ex- perimentally demonstrated that ASTE can be better solved using seq.-to-seq. generative approaches.
that while a sentence can have multiple aspects, the combination of (sen;, Pij) will have single senti- ment label Sij- We denote data point (sen;, Pij) as i and the corresponding sentiment label as yi- Further, to train the model, we can now derive mul- tiple data points from sentences with more than one aspect. This alleviates to some extent the large amount of data required to effectively (pre)train a model using contrastive learning (Li et al., 2021b),
Given the choice of our fine-tuning framework, we want both the encoder as well as the decoder to take advantage our proposed pre-training strat- egy. Therefore, we do not obtain the aspect-aware sentiment representations from input sentences us- ing only an encoder (although possible). Rather, we continually pre-train the full model (from its generic pre-trained checkpoint) using aspect-based prompts. The decoder is expected to learn from the contextualized representations of input sentences as produced by the encoder and decode aspect-aware sentiment representations for the corresponding [MASK] tokens based on the prompts receives. Supervised contrastive learning is performed on these aspect-centric sentiment embeddings.
As depicted in Fig. [(a), we pass the input sen- tence through the encoder, and provide an aspect- based prompt to the decoder. The prompt consists of a [MASK] that masks the sentiment associated with the aspect. The decoder-ggeerated output for the [MASK] token forms the aspect-centric senti- ment representation 2; corresponding to the data point Z;. The supervised contrastive loss on the batch I is defined follows:
eaplz: + Zp/T)
noup
log
PQ
EeEAio expp 25 2u/T
PE PEP
(1)
Here, index i represents the anchor. A(i) = I represents the set of all indices except the anchor. P(i) {p E A(i) 3p y1 is the set of indices of all positives (same sentiment label) distinct from and P(i) its cardinality. The symbol denotes the inner dot product between two embeddings, and E R is a scalar temperature parameter.
In order pre-train the model, first we obtain aspect-aware sentiment representation(s) from a given sentence sen; using asseet-based prompts Pij for each aspect term ay. to be noted here

---

2.2
Multi-task Approach For ASTE
mention detection, modeled as a token-wise binary classification task, to improve the performance of the main task of autoregressive entity linking, for- mulated as a "language generation task". We have a similar setting, where our main task of ASTE fol- lows a generative paradigm, whereas, as depicted in Fig. 1(b), we formulate OTD as a sequence-tagging task using the BIO scheme. We hypothesize that the opinion term detection (OTD) module will help to better detect the opinion span boundaries which in turn affects the sentiment prediction. Formally, for cach token tok; E z, the opinion tagger takes as input the contextualized token embedding gener- ated by the encoder, and performs a 3-way classifi- cation task with the classes being B- B--beinnnig of the span, I-inside the span, O-outside the span. The module is trained by minimizing the Cross Entropy loss (Lord) between the true and predicted labels. Our ablation results reported in Sec. 4 further jus- tify the importance of this module.
After being pre-trained, the encoder-decoder frame- work now needs to be fine-tuned for the ASTE task. For building our fine-tuning architecture, we leverage multi-task learning (MTL) which has been successfully used across range of ABSA tasks (Collobert and Weston, 2008; He et al., 2019; Gao et al., 2022) to improve the performance of the main task by designing and training related auxil- iary tasks jointly. Designing appropriate auxiliary tasks is also challenging since MTL is not guaran- teed to always improve the main task performance (Martínez Alonso and Plank, 2017). Next, we de- scribe how T5 is trained for the main task of ASTE, before elaborating on the motivation and working of two auxiliary modules as depicted in Fig. 1(b). Corresponding to each sentence æ being passed as input to the encoder, first we construct the target sequence to be generated by the decoder. Let T= (ty tz (aspy.opin,, sy))/21 be the set of opinion triplets associated with æ. The linearized target sequence y, as depicted in Fig. ((b), takes the form as reported in Table 2. The target construc- tion algorithm is presented in Algorithm 1 (refer A.1). Let denote the encoder-generated contex- tualized representation of At the i-th time step, the decoder output y Dec, is computed based on e and the previous outputs Vei- Proba- bility distribution for the next token is obtained as:
2.2.2 Triplet Count Estimation (TCE)
During fine-tuning of 'T for the ASTE task, the tar- get sequences explicitly guide the decoder, through supervision, on how many triplets generate. order to further augment this process, we intro- duce the auxiliary task of triplet count estimation (TCE). Part of our motivation comes from (Mrini et al., 2022) where the second auxiliary task of en- tity match prediction is modeled as a classification task. Please note however that the exact number of triplets in a test sentence not known priori. Hence, we design TCE as simple regressor, con- sisting of two layers of fully connected networks (FCN). Specifically, we take the encoder-gent greoder-generated sentence embedding as input (768-dim.) and pass it through the first FCN layer consisting of 128 neu- rons. The outputs of this layer are passed through the second layer of FCN which consists of single neuron. The regressor is trained to predict the num- ber of triplets associated with the sentence by minimizing the Mean Squared Error loss (Crck). It is to be noted here that the TCE module, being a regressor, generates float values. For inference, we round off the output to the nearest integer (and < integer > .5 to < integer >). However, there is no direct influence of the TCE output the T5 decoder in the absence of any explicit connection between the two. Please note that this architectural choice is consistent with the literature on multi-task lcarning unless the main and auxiliary modules are trained in a hierarchical setup (Sanh et al., 2019).
Po( 1|,, (c)+1) so coftmax(W" (W"
(2)
Here, is initialized with parameter weights ob- tained after pre- pre-training the model using contrastive learning. W maps yi to a logit vector which is then used to calculate the probability distribution over the whole vocabulary set. It is to be noted here that the tokens <aspect>, <opinion>, and <sentiment> are added to the vocabulary at the time of train- ing, and their embeddings are learnt from scratch. Finally, the model parameters are fine-tuned on the input-target pairs by minimizing the negative log-likelihood Po(u)e) (denoted LED) as follows:
LED = -log po (yle) = log vo(
where n is the length of the target sequence y.
2.2.1 Opinion Term Detection (OTD) The motivation behind including this module comes from (Mrini et al., 2022) where the authors introduce a similar auxiliary module called entiry

---

Datasets
#5
POS
NEU
NEG
pre-training dataset. This results in total of 5,039 train data points (refer Scction 2.1) from 3,634 sentences. Please note that we do not need test data in the pre-training phase. Also, different from Li et al. (2021b), we do not use any exter- nal data for performing supervised contrastive pre- training. Pre-trained 15-base! was used to initialize the model weights. We (pre)train the TS encoder- decoder framework for 14 epochs using AdamW optimizer (Loshchilov and Hutter, 2017) with a learning rate of 2e-7 and a batch size of 16. The temperature parameter J was set to 0.07.
Train
906
817
126
517
Lapl4
Dev
219
169
36
141
Test
328
364
63
116
Train
1266
1692
166
480
14Res
Dev
310
404
54
119
Test
492
773
66
155
Train
605
783
25
205
15Res
Dev
148
185
11
53
Test
322
317
25
143
Train
857
1015
50
329
16Res
Dev
210
252
11
76
Test
326
407
29
78
Table ASTE-VE dataset statistics. #S denotes the no. of sentences, 'POS', 'NEU", and 'NEG' denote the no. positive, neutral, and negative triplets respectively.
3.2.2 Fine-Tuning
CONTRASTE-MIL. contains around 222 million trainable parameters. For each of the datasets, we respectively fine-tune the pre-trained model weights for the downstream ASTE task using AdamW optimizer with learning rate of le-4 for 14Res and 16Res, and 3e-4 for 15Res and Lap14. batch size of 16 was used for all datasets. Fol- lowing Mrini et al. (2022), we optimize the auxil- iary task weights, a (OTD), and /3 (TCE) for each dataset. As shown in Fig. 3, we start by optimiz- ing a with /3 set to 0.4. We then optimize 3 given the optimal a values, One can visibly observe that the ASTE performance varies with changing task weights. For each dataset, we obtain dif- ferent set of optimal a and | values based on the highest ASTE F1 scores on the respective val sets. Fig 3 shows the optimal weights on all for ASTE- Data-V2 datasets; a = 1.0, 3 = 0.4 on Lap14, = 0.2, 3 = 0.6 on 14Res, a - 0.8, B = 0.4 on 15Res, and a 0.8, 3 0.8 16Res
Also, we do not post-process the decoder-generated sequence match the same number of triplets as predicted by the TCE module before calculating the final results. The only implicit guidance be- tween T5 and TCE expected through the joint optimization of loss functions as described next. Our ablation results (Sec. 4) justify the advantage of TCE in improving ASTE performance.
2.2.3 Joint Training
Our base model, CONTRASTE-Bass is trained by optimizing the encoder-decoder loss LED only. The full model, CONTRASTE-MTL is jointly trained in a multi-task setup by minimizing the combined loss L as follows:
== L + a LOTD + B -.CE
(4)
a and 3 are the weight coefficients assigned to the OTD loss LOTD. and TCE loss LTCE respectively.
3 Experiments
Each of our models was trained for 20 epochs and the model instance corresponding to the best val Fl score was used to evaluate the test set. We report the median scores over five runs of the exper- iments. Each pre-training epoch took 10 minutes and each fine-tuning epoch took minute on 15Res and 2 minutes on the other three datasets. All our experiments were run on Tesla P100-PCIE 16GB GPU. We make our codes publicly available.?
3.1 Datasets & Evaluation Metrics
We evaluate CONTRASTE on four ASTE datasets (ASTE-Data-V2) released by Xu et al. (2020), which includes one dataset from the laptop domain and three datasets from the restaurant domain. The statisties of all the datasets are shown in Table 3. Following prior works, we report precision, re- call and Fl scores to evaluate and compare meth- ods on the ASTE task. A predicted triplet is con- sidered correct if all its predicted elements exactly match with those of a ground-truth opinion triplet.
3.3 Baselines
We compare our proposed approach with sev- eral state-of-the-art ASTE baselines which can be broadly grouped into the following five categories: Pipeline: CMLA, RINANTE (Dai and Song. 2019) and Li-unified-R (Li et al., 2019) are
3.2 Experimental Setup
3.2.1 Pre-Training
For this, we combine the train data from all four ASTE-DATA-V? datasets (Table 3) to prepare our
hhtpss//huggiinffccc.osss.s.

---

Model
14Res
15Res
16Res
Lapl4
PARAPHRASE
0.715
0.621
0.719
0.605
ASTE-Base
0.720
0.634
0.722
0.608
w/ SCL-Sent.
0.722
0.645
0.724
0.611
CONTRASTE-BaS
0.728
0.648
0.730
0.614
Table 4: Test Fi scores on ASTE. Comparing PARA-
PHRASE with our non-multi-task model variants.
Neggove
Befbee Costractive Pre-training
Ib) Afferr Cantradive Pre-drainnnng
pipeline methods to co-extract aspects and opin- ion terms. CMLA+, RINANTE+, Li-unified-E and Peng-two-stage are improved versions pro- posed by (Peng ct al., 2020a) to jointly extract the aspects, opinions, and corresponding sentiments. Tagging-based: OTE-MTL (Zhang et al., 2020) propose multi-task framework to jointly extract the three sentiment terms. JET-BERT (Xu et al.. 2020) is an end-to-end approach that proposes a novel position-aware tagging scheme. GTS- BERT (Wu et al., 2020) models ASTE as a grid- tagging task. EMC-GCN (Chen et al., 2022a) proposes an Enhanced Multi-Chun Channel GCN net- work model the relation between words.
Figure 2: t-SNE visualization of decoder-generated [MASK] token embeddings from aspect- based prompts derived from 15Res val set. Our SCL objective encour- ages the decoder to produce discriminable representa- tions of different sentiment polarities.
our proposed pre-training strategy (refer Section 2.1) and then fine-tuned for ASTE using our pro- posed templates (refer Table 2). We denote ASTE- Base as the variant of CONTRASTE-BaLe Base without the pre-training part. In essence ASTE-Base is di- rectly comparable PARAPHRASE (Zhang al., 2021a), the difference being the templates used for fine-tuning. As previously discussed, the tem- plates used in PARAPHRASE do not generalize well across ABSA tasks, and often lead to semanti- cally meaningless or wrong targets (refer Table 2). Our prompt-based templates, on the other hand, can casily be extended across tasks, such ACOS, detailed in Section 4. Fine-tuning the T5 encoder- decoder framework with our templates also results in better ASTE performance as reported in Table 4. One can observe that ASTE-Base outperforms MARAPHRASEE with overall 0.9% Fi gains across all four ASTE benchmark datasets. Detailed results for all compared models are reported in Table 5.
Span-based: Span-ASTE (Xu al., 2021) con- siders the span-level interactions of targets and opinions while predicting the sentiment. SBSK- ASTE (Feng et al., 2022) uses abundant syntax Knowledge to improve ASTE. Span-BiDir (Chen et al., 2022b) is recent approach that uses span-level bidirectional network utilize all pos- sible spans for extracting tuples bidirectionally. Generative: PASTE (Mukherjee et al., 2021) is tagging-free free decoding scheme using pointer networks for ASTE.
(Yan et al., 2021) unifies all ABSA tasks with a gen- erative framework using BART. GAS (Zhang et al., 20) proposes ba-d solution with linearized templated targets. PARAPHRASE (Zhang et al., 202 proposes to decode triplets templated natural language paraphrases.
3.5 Comparison of Supervised Contrastive Pre-Training Approaches
As discussed in Section 2.1, design novel aspect- based prompts to pre-train our encoder-decoder framework by performing supervised contrastive learning (SCL) on menodder-generate aspect-aware sentiment embeddings of [MASK] tokens. In or- der to qualitatively understand the effect of such pre- are-trriiiig on the aspect- based sentiment learning ability of our framework, we show in Fig. 2, a t-SNE (van der Maaten and Hinton, 2008) visual- ization of these embeddings on all the aspect-based prompts derived from all four ASTE-Data- V2 test sets. We observe that the positive, and negative sen- timent embeddings are better clustered and more neatly separated from each other after pre-training.
MRC-based: BMRC (Chen al., 2021) pro- poses a novel method to solve ASTE as Bidi- rectional Machine Reading Comprchension task. Among these, PARAPHRASE is closest to fine-tuning approach, and one of the strongest base- lines. Next, we demonstrate the better advantage of our prompt-based templates over the ones used in PARAPHRASE to fine-tune the model for ASTE,
3.4 PARAPHRASE vs. Our Templates
CONTRASTE-BE Base refers to our base (non-multt task) model variant which is first pre-trained using

---

14Res
15Res
16Res
Lapl4
Model
P.
R.
P.
R.
Fτ
P.
R.
F1
P.
R.
F1
CMLA+
0.392
0,471
0.428
0.346
0.398
0.370
0.413
0.421
0.417
0.301
0.369
0.332
RINANTE+
0.314
0.394
0.350
0.299
0.301
0,300
0.257
0.223
0.239
0.217
0.187
0.201
Li-unified-R
0.410
0.674
0.510
0.447
0.514
0.478
0.373
0.545
0.443
0.406
0,443
0.423
Peng-rwo-stage
0.432
0.637
0.515
0.481
0.575
0.523
0.470
0.642
0.542
0.374
0,504
0.429
ChatGPT &
0.513
0.629
0,565
0.419
0.606
0.495
0.449
0.617
0.520
0.351
0,489
0.409
OTE-MTL
0.630
0.551
0.587
0.579
0.427
0.489
0.603
0.534
0.565
0.492
0,405
0.451
JET-BERT
0.706
0.559
0.624
0.645
0.520
0.575
0.704
0.584
0.638
0.554
0.473
0.510
PASTE
0.648
0.638
0.643
0.583
0..667
0.575
0.655
0.644
0.650
0,550
0,516
0.532
GTS-BERT
0.680
0.676
0.678
0.627
0.555
0.589
0.654
0.680
0,667
0.561
0.530
0.545
GAS
0.650
0.695
0.672
0,561
0.618
0,588
0.661
0.687
0.674
0.571
0,540
0,555
Unified-BART-ABSA
0.655
0.650
0.653
0.591
0.594
0.593
0.666
0.687
0,676
0.614
0.562
0.587
BMRC
0.756
0.618
0.680
0.685
0.534
0.601
0.712
0.611
0.658
0.706
0.490
0,578
EMC-GCN
0.712
0.724
0.718
0.615
0.625
0,619
0.656
0.713
0.683
0.617
0,563
0,588
Span-ASTE
0.729
0.709
0.719
0.622
0.645
0.633
0.695
0.712
0.703
0.634
0.558
0.594
PARAPHRASE
0.711
0.719
0.715
0.604
0.639
0,621
0.701
0.739
0.719
0.634
0.578
0.604
SBSK-ASTE
0.746
0.715
0.730
0.653
0.637
0.645
0.708
0.720
0.714
0.656
0.565
0.607
Span-BiDir
0.764
0.724
0.743
0.699
0.604
0.648
0.716
0.726
0.721
0.657
0,599
0.627
ASTE- Basse
0.718
0.721
0.720
0.616
0.654
0.634
0.695
0.751
0.722
0.635
0,584
0.608
CONTRASTE-Bas
0.724
0.732
0.728
0.626
0.672
0.648
0.721
0.739
0.730
0.639
0.591
0.614
CONTRASTE-ME 0.736
0.744
0.740
0.653
0.667
0.661
0.722
0.763
0.742
0.642
0.617
0.629
Table 5: Comparative results on the ASTE-Data- V2 (Xu et al., 2020).
denotes that the results are retrieved from
Xu al. (2020), & ChatGPT results are obtained using 100-shot In Context Learning (ICL) prompts. The results for all other methods were reproduced using released codes and original parameters. The highest Fl scores on each dataset are highlighted bold. The second highest scores are underlined.
Thus, our SCL-based pre-training objective helps in improving the performance on ABSA tasks.
suitability of performing supervised contrastive pre- training on aspect-centric sentiment embeddings, since in ABSA, the sentiments are defined at an aspect level and not at the sentence level.
Next, we wanted to compare our pre-training strategy with the one used in existing ABSA works. While Li et al. (2021b) uses SCL for pre-training, Liang et al. (2021), and Ke al. (2021) use it for fine-tuning their respective models. However, dif- ferent from us, all of them apply SCL on sentence- level sentiment representations of sentences. In or- der to replicate their methodology, we pre-train our mncodrr-deooder framework by applying SCL on mean-pooled representations of sentences from the final layer of encoder. For this, we collected total of 3358 data points (sentences containing triplets with the same sentiment polarity) from 3,634 sen- tences combining all four train datasets. Model weights were initialized with pre-trained 15-base, and the framework was (pre)trained for 14 epochs using AdamW optimizer with a learning rate of 2e-5, batch size of 16, and J = 0.07.
3.6 Main Results
We report the comparison of our model variants with all considered baselines in Table 5. The majority of the baselines including OTE-MTI, JET-BERT, PASTE, GTS-BERT, BMRC, Span- ASTE, EMC-GCN, SBSK-ASTE, and Span-BiDir use BERT (Devlin et al., 2019) as their backbone. Unified-BART-ABSA uses BART (Lewis et al., 2020). GAS and PARAPHRASE are based on T5 encoder-decoder framework and hence are most similar to our approach. PARAPHRASE is our strongest basbd baseline. SBSK- ASTE and Span-BiDir are very recent span-based techniques and they qualify to be our strongest two baselines. We find that ASTE-Base, our non-multi- task non-pre-trained model outperforms PARA- PHRASE with overall 0.9% F, gains, however, fails to beat SBSK-ASTE and Span-BiDir. This observation may be attributed better feature learning strategies employed by these two meth- ods. We find that our proposed SCL-based pre- training approach hugely improves the perfor- mance of ASTE-Base Base, as CONTRASTE- -aae out-
We fine-tune ASTE-base from this pre-trained checkpoint (settings discussed in Section 3.2) re- spectively for each of the datasets and report our results in Table 4 (row ASTE-Base w/ SCL-Sent.). We observe that CONTRASTE-Bae outperforms ASTE-Bare w/ SCL-Sent. with overall 0.6% % im- provement in F, scores. This establishes the better

---

CON
OTD
TCE
Triplet
Aspect
Opinion
Sentiment
Model
Res15
Res16
Rest-ACOS
Lap-ACOS
x
X
0.671
0,820
0.815
0.753
CharGPT
0.262
0.386
0.237
0.824
0.820
0.765
PARAPHRASE
0.468
0.578
0.592
0.429
1
/
0.678
0.827
0.827
0.762
0.685
0.832
0.328
0.770
ACOS-Ban
0.458
0.583
0.597
0.431
x
0.682
0.836
0.842
0.776
ACOS-Contra
0.478
0.598
0.605
0.446
0.692
0.840
0.848
0.784
Table 7: Test Fi scores on ACOS, Comparing ChatGPT and PARAPHRASE with our ACOS model variants.
Table 6: Ablation Results. We report Fl scores for Triplet, Aspect, and Opinion predictions, and Sentiment accuracies for correctly predicted tuples (asp. opin).
rectly predicted opinion tuples (aspect term, opin- ion term) are also reported. The first row corre- sponds ASTE- and the last row corresponds to CONTRASTE-MILL results from Table
performs SBSK-ASTE with overall 1.4% F1 gains, while performing comparably with Span-BiDir. Finally, CONTRASTE-MIL comfortably outper- forms PARAPHRASE with overall 4.4% Fj gains, SBSK-ASTE with overall 2.8% Fi gains, and Span-BiDir with overall 1.2% % Fi gains, to achieve new stafe-of-the-art ASTE results.
From Table 6, we observe (bottom to top) that removing TCE and OTD degrades the triplet FI scores by 1.3%, and 0.8% respectively, thereby highlighting their impact on the overall task. Re- moving TCE results in more spurious triplets being generated, which in turn hampers the scores. Re- moving OTD especially affects the opinion term ex- traction (F1 + 2.4%) and sentiment prediction (acc + 1.8%) performance. Removing CON substan- tially affects all the scores (triplet, aspect, opinion FI scores + 1.9%, 1.6%, 2.5% respectively, senti- ment acc 2.9%), thereby demonstrating the advan- tage of contrastive pre-training using our proposed approach. This observation is further strengthened when we compare the second row, corresponding to CONTRASTE- -Baee, with the first row, corre- sponding to ASTE-Base and see a huge jump in sentiment acc (t 1.6%). Finally, removing all com- ponents results in significant drop across all the scores, thereby establishing the importance of all proposed components of CONTRASTE-MTLL.
4 Analysis
4.1
Experiments With ChatGPT
Natural Language Processing, in recent times, has been revolutionized by the evolution of Large Lan- guage Models (LLMs) such as GPT-3 (Brown et al., 2020). ChatGPT (OpenAI, 2023b), powered by GPT-3.5 and GPT-4 (OpenAl, 2023a), has pio- neered the excitement around LLMs by achieving remarkable zero-shot and few-shot in-context learn- ing (ICL) (Brown et al., 2020) results for unseen NLP tasks, without any parameter updates.
In this work, we investigated how well can Chat- GPT perform on ABSA tasks. We experimented with 4 different kinds of zero-shot and few-shot prompts as detailed in Sec. A.2. For each task, our best test set results are obtained when we prompt ChatGPT with task-specific instructions followed by 100 randomly selected (sentence, target out- put) samples from the corresponding training set. ASTE results obtained with ChatGPT are reported in Table 5. Compared to our CONTRASTE-MTL. re- sults, we observe huge gap in performance. This demonstrates that ChatGPT is far from producing SOTA ASTE results (refer Sec. A.2 for details).
4.3 Qualitative Analysis
Table compares the model predictions of PARA- PHRASE and CONTRASTE- for few test sentences. The first example highlights the advan- tage of both TCE and OTD as we correctly predict both the number of triplets, as well as the triplets themselves (and hence the opinion terms) whereas PARAPHRASE misses out on both. The second example further establishes the importance of OTD as we correctly predict the opinion term boundaries whereas PARAPHRASE fails to do so.
4.2 Model Ablations
Contrastive pre-training (CON), OTD, and TCE are the three crucial components of CONTRASTE- time
4.4 Advantage of Pre- Tre-Training On ACOS
MTL. Here, we ablate one component and report in Table 6 the Fl scores correspond- ing to the triplet, aspect term, and opinion term predictions averaged over all four datasets. Senti- ment prediction accuracies corresponding to cor-
We consider the task of Aspect Category Opin- ion Sentiment (ACOS) quad prediction (Cai et al., 2021; Zhang et al., 2021a) in order to understand whether our proposed bCL-based pre-training tech-

---

SENTENCR
TRIPLETS
GOLD: place Gorgeous ; POST place iddall POS
Gorgeous place ideal for a romantic dinner.
PARAPHRASE: place romantic POS
CONTRASTE-MTL. place ; Gorgeous POS I place: : ideal POS
GOLD: manager: not even apologetic ; NEG PARAPHRASE: manager; ;
I complained the manager, but he not even apologetic.
apologetic; NEG
CONTRASTE-MTL manager: not even apologetic NEG
Table 8: Few test set sentences; their gold triplets, & predictions made by PARAPHRASE and CONTRASTE-MTL.
nique can improve the downstream performance of more difficult ABSA tasks as well. Here, given a sentence. for example, "the spicy tuna roll was un- usually good", the task is to extract the (aspect, cat- egory, opinion, sentiment) quad ('spicy tuma roll', 'food quality', 'unusually good', POS). Also, there could be multiple such quads in given sentence.
form CL on aspect-aware sentiment representations of masked opinion terms expressing the sentiment, Liang et al. (2021) try to distinguish between aspect-invariant and aspect-dependent sentiment features. Further, their data augmentation strategy masks the aspect terms, whereas we mask the senti- ment. Finally, they apply contrastive learning only to fine-tune the BERT encoder, whereas we use SCL to pre-train an encoder-decoder setup.
We perform our experiments on four datasets, Restaurant-ACOS, Laptop-ACOS released by Cai et al. (2021), and Rest15, 5, Restló 6 released by the au- thors of PARAPHRASE (Zhang et al., 2021 and report our results in Table 7. Experimental details be found in Sec. A.3. Here, ACOS-Base refers T5 trained with our proposed fine-tuning tem- plates (updated for ACOS). ACOS- CEcos-Contra refers to the fine-tuning of ACOS-Base from a check- point pre-trained using our proposed SCL-based strategy (refer Sec. 2.1). We observe that while the performance of that of KARAPHRASE. supervised contrastive pre- training of the model hugely improves the scores as ACOS-Contra outperforms both PARAPHRASE, and ACOS-Bass with overall 2.9% F1 gains. Also, ACOS-Contra substantially outperforms ChatGPT. Baselines and comparative results for Target As- pect Sentiment Detection (TASD), and Aspect Ex- traction and Sentiment Classification (AESC) are respectively reported in Sections A.4 and A.5.
Ke et al. (2021) apply contrastive learning in a continual learning setup with very different objec- tives of knowledge transfer across tasks, and knowl- edge distillation from old tasks the new task. Morcover, all these works tackle a relatively casier downstream ABSA task of Aspect Sentiment Clas- sification (ASC), where the goal is to determine the sentiment, given the sentence and a specific aspect.
Also, all of them are encoder- based approaches. The novelty of our approach, therefore lies in de- signing aspect-based prompts, obtaining aspect- aware sentiment representations of masked senti- ment terms, and applying contrastive learning on these embeddings (pre) an encoder-decoder framework for the downstream ASTE task.
is comparable to
6 Conclusion
We propose a novel strategy to continually pre-train T5, from its publicly available generic pre-trained checkpoint, for ABSA tasks. For this, given sen- tence, first, derive aspect-based prompts with corresponding sentiments masked. We then ap- ply supervised contrastive learning (SCL) on the decoder-generated aspect-level sentiment embed- dings of the masked tokens. Compared to perform- ing SCL on sentence-level sentiment embeddings, we show that our strategy results in a better down- stream performance for ABSA tasks such as ASTF, ACOS, TASD, and AESC. Also, do not any external data for pre-training. For task-specific fine- tuning, we propose generic placeholder-batt tem- plates to train T5. Finally, we present our multi-task model, CONTRASTE-MTT that achieves SOTA re- sults on the majority of ASTE benchmark datasets.
5 Related Works
Supervised Contrastive Learning (SCL) has been explored previously in the ABSA domain with very different objectives and approaches than ours. et al. (2021b) use supervised contrastive pre- training to effectively learn implicit sentiments. However, their approach is not optimal as they apply contrastive learning on sentence-level sen- timent representations of sentences, whereas in ASTE (or any other ABSA task), sentiments are de- fined at an aspect-level. Additionally, they rely on external large-scal, scale sentiment-annotated corpora to learn sentiment knowledge. We, differ from them as we do not use any external data. Also, we per-

---

7 Limitations
conditional text generation for aspect-based senti- ment analysis. In Proceedings of the 35th Pucific Asia Conference on Langwage, Information and Com- putation, pages 119-129, Shanghai, China. Associa- tion for Computational Lingustics.
It requires a substantial amount of data to effec- tively apply contrastive learning (Li et al., 2021b) even for continually pre-training an already pre- trained (generic) encoder-decoder model such as T5. However, in this work, we limited ourselves to the training sets of existing ASTE benchmarks. Us- ing our proposed aspect-based prompts, we could obtain more data points than the actual number of sentences, and they were sufficient enough for us to achieve new stateoof-the-are results. The impact of pre-training with more data however remains to be investigated.
Hao Chen, Zepeng Zhai, Fangxiang Feng, Ruifan Li, and Xiaojie Wang. 2022a. Enhanced multi-channel graph convolutional network for aspect sentiment triplet extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Pupers), pages 2974-2985, Dublin, Ireland. Association for Computational Lin- guistics.
Shaowei Chen, Yu Wang, Jie Liu, and Yuelin Wang. 2021. Bidirectional machine reading comprehension for aspect sentiment triplet extraction. In Proceed- ings Of The AAAI Conference On Artificial Intelli- gence, volume 35, pages 12666-1266 12674.
While preparing the pre-training data, we com- bined data from both domains, that is, laptop and restaurant. It was done mainly because our exper- iments with pre-training the model with a limited number of laptop data points could not give us good results (please note that there are three restaurant datasets vs one laptop dataset in the ASTE-Data- V2 benchmark). This raises an important question regarding how much data is sufficient enough to ef- fectively pre-train the model. Running pre-training experiments with different fractions of available data is something that we have not explored here. Also, the applicability of our proposed scheme in cross dross-domain settings remains to be investigated.
Yugi Chen, Keming Chen, Xian Sun, and Zequn Zhang. 2022b. A span-level bidirectional network for aspect sentiment triplet extraction.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sehastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin- odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury. Jacob Austin, Michael Isard, Guy Gur-Ari, Pengeheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan. Shivani Agrawal, Mark Omernick, An- drew M. Dai, Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Sacta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language mod- eling with pathways.
References
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voo Voss. Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz. Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever. and Dario Amodei, 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. Proceed- ings of the 25th International Conference on Machine Learning, ICML '08, page 160-167, New York, NY, USA. Association for Computing Machinery.
Hongjie Cai, Rui Xia, and Jianfei Yu. 2021. Aspect- salegorr-opinioo--settment quadruple extraction with implicit aspects and opinions. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 340-350, Online. Association for Computational Linguistics.
Hongliang Dai and Yangqiu Song. 2019. Neural aspect and opinion term extraction with mined rules as weak supervision. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics. pages 5268-5277, Florence, Italy. Association for Computational Linguistics.
Siva Uday Sampreeth Chebolu, Franck Dernoncourt, Nedim Lipka, and Thamar Solorio. 2021. Exploring
Jacob Devlin, Ming-Was Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of

---

deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume 1 (Long and Short Pupers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
pages 7871-7880, Online, Association for Computa- tional Linguistics.
Chengxi Li, Feiyu Gao, Jiajun Bu, Lu Xu, Xiang Chen, Yu Gu, Zirui Shao, Qi Zheng, Ningyu Zhang, Yong- pan Wang, and Zhi Yu. 2021a. Sentiprompt: Senti- ment knowledge enhanced prompt-tuning for aspect- based sentiment analysis.
Lingcong Feng, Biqing Zeng, Lewei He, Mayi Xu, Huimin Deng, Pengfei Chen, Zipeng Huang, and Weihua Du. 2022. Improving span-based aspect sen- timent triplet extraction with abundant syntax knowl- edge.
Xin Li, Lidong Bing, Piji Li, and Wai Lam. 2019. A unified model for opinion target extraction and target sentiment prediction.
Zhengyan Li, Yicheng Zou, Chong Zhang, Qi Zhang. and Zhongyu Wei. 202 Learning implicit senti- ment in aspect-based sentiment analysis with super- vised contrastive pre-training. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 246-256, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Tianhao Gao, Jun Fang, Hanyu Liu, Zhiyuan Liu. Chao Liu. Pengzhang Liu. Yongjun Bao, and Weipeng Yan. 2022. LEGO-ABSA: A prompt- prompt-based task assem- blable unified generative framework for multi-task aspect-based sentiment analysis. In Proceedings of the 29th International Conference on Computational Linguistics, pages 7002-7012, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Bin Liang, Wangda Luo, Xiang Li. Lin Gui, Min Yang. Xiaoqi Yu, and Ruifeng Xu. 2021. Enhancing aspect- based sentiment analysis with supervised contrastive learning. In Proceedings of the 30th ACM Inter- national Conference on Information &amp; Knowl- edge Management, CIKM '21, page 9242-3247, New York, NY, USA. Association for Computing Machin- ery.
Ridong Han, Tao Peng. Chaohao Yang. Benyou Wang.
Lu Liu, and Xiang Wan. 2023. Is information extrac- tion solved by chatgpt? an analysis of performance, evaluation criteria, robustness and errors.
Ruidan He. Wee Sun Lee, Hwee Tou Ng. and Daniel Dahlmeier. 2019. An interactive multi-task learning network for end-to-end aspect-hased sentiment anal- ysis. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pages 504-515, Florence, Italy. Association for Computa- tional Linguistics.
Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization.
Haoran Lv, Junyi Liu, Henan Wang, Yaoming Wang, Jixiang Luo, and Yaxiao Liu. 2023. Efficient hy- brid generation framework for aspect-based senti- ment amalysis. In Proceedings of the 17th Conference of the European Chapter of the Association for Com- putational Linguistics, pages 1007-1018, Dubrovnik, Croatia, Association for Computational Linguistics.
Pere-Lluis Huguet Cabot and Roberto Navigli. 2021. REBEL: Relation extraction by end-to-end language generation. In Findings of the Association for Com- putational Linguistics: EMNLP 2021, pages 2370- 2381, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yue Mao, Yi Shen, Jingchao Yang, Xiaoying Zhu, and Longjun Cai. 2022. Seq2Path: Generating sentiment tuples as paths of a tree. In Findings of the Asso- ciation for Computational Linguistics: ACI. 2022, pages 2215-2225 Dublin, Ireland. Association for Computational Linguistics.
Zixuan Ke, Bing Liu, Hu Xu, and Lei Shu. 2021. CLAS- SIC: Continual and contrastive learning of aspect sentiment classification tasks. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pages 6871-6883, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Héctor Martínez Alonso and Barbara Plank. 2017. When is multitask learning effective? semantic se- quence prediction under varying data conditions. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Lin- guisties: Volume 1, Long Papers, pages 44-53, 53, Valen- cia, Spain. Association for Computational Linguis- tics.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Su- pervised contrastive learning. Advances in Neural Information Processing Systems, 33: 186111111.773
Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, AbdeIrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequuee
Khalil Mrini, Shaoliang Nie, Jiatao Gu, Sinong Wang. Maziar Sanjabi. and Hamed Firooz. 2022. Detec- tion, disambiguation, re- re-ranking: Autoregressive en- tity linking as a multi-task problem. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1972-1983, Dublin, Ireland. Association for Computational Linguistics.
pre-training for natural language generation, translation, and com- prehension. Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics,

---

Rajdeep Mukherjee, Tapas Nayak, Yash Butala, Sourangshu Bhattacharya, and Pawan Goyal. 2021. PASTE: A tagging-free decoding framework using pointer networks for aspect sentiment triplet extrac- tion. In Proceedings of the 2021 Conference on Em- pirical Methods in Natural Language Processing, pages 1279-9291, Online and Punta Cana, Domini- can Republic. Association for Computational Lin-
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86):2579-2655.
Hai Wan, Yufei Yang, Jianfeng Du, Yanan Liu, Kunxun Qi, and Jeff Z. Pan. 2020. spret-eenpettttteeimmtnttt joint detection for aspect-based sentiment analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):9122 9129..
guistics.
Zhen Wu, Chengcan Ying, Fei Zhao, Zhifang Fan, Xinyu Dai, and Rui Xia. 2020. Grid tagging scheme for aspect-oriented fine- geeegga opinion extraction. In Findings of the Association for Computational Lin- guisties: EMNLP 2020, pages 2576-2585, Online. Association for Computational Linguistics.
OpenAI. 2023a. Gpt-4 technical report.
OpenAL. 2023b. Introducing chatgpt.
Haiyun Peng, Lu Xu, Lidong Bing, Fei Huang, Wei Lu, and Luo Si. 2020a. Knowing what, how and why: A near complete solution for aspect-based sentiment analysis. In AAAI.
Lu Xu, Yew Ken Chia, and Lidong Bing. 2021. Learn- ing span-level interactions for aspect sentiment triplet extraction. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 4755 4766, Online. Association for Computa- tional Linguistics.
Haiyun Peng, Lu Xu, Lidong Bing, Fei Huang, Wei Lu, and Luo Si. 2020b. Knowing what, how and why: A near complete solution for aspect-based sentiment analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05)x8600866) 8607.
Lu Xu, Hao Li, Wei Lu, and Lidong Bing. 2020. Dosition-awars tagging for aspect sentiment triplet extraction. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 2339-2349, Online. Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang. Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-texx text trans- former.
Victor Sanh. Thomas Wolf, and Sebastian Ruder. 2019. hierarchical task approach for learning em- beddings from semantic tasks. In Proceedings of the Thirty-Thii Third AAAI Conference on Artificial Intelli- gence and Thirty-First Innovative Applications of Ar- tificial Intelligence Conference and Ninth AAAI Sym- posium on Educational Advances in Artificial Intelli- gence, AAAI' 19/AAT 19. AAAI Press.
Hang Yan, Junqi Dai, Tuo Ji, Xipeng Qiu, and Zheng Zhang. 2021. unified generative framework for aspect-based sentiment analysis. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1sh International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2416-2429, Online. Association for Computational Linguistics.
Chen Zhang, Qiuchi Li, Dawei Song, and Benyou Wang. 2020. A multi-task learning framework for opinion triplet extraction. In Findings of the Association for Computational Linguisties: EMNLP 2020. pages 819-828, Online. Association for Computational Lin- guistics.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almabairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava. Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, An- thony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Maric-Anne Lachaux, Thibaut Lavril, Jemya Lee, Di- ana Liskovich, Yinghai Lu, Yuming Mao, Xavier Mar- tinet, Todor Mihaylov, Pushkar Mishra, Igor Moly- bog, Yixin Nie, Andrew Poulton, Jeremy Reizen- stein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subrama- nian, Xiaoqing Ellen Tan, Binh Tang, Ross Tay- lor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Ro- driguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open foundation and fine- tuned chat models.
Wenxuan Zhang, Yang Deng, Xin Li, Yifei Yuan, Li- dong Bing. and Wai Lam. 2021a. Aspect sentiment quad prediction as paraphrase generation. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 9209- 9219, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Wenxuan Zhang, Xin Li. Yang Deng, Lidong Bing, and Wai Lam. 202 Towards generative aspect-based sentiment analysis. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 2: Short Papers), pages 504-514 510, Online. Association for Computational Linguistics.

---

14Res
15Res
16Res
Lapl4
Model
P.
Fi
P.
R.
F₁
P.
R.
F
R.
F₁
5-shot ICL (Ham et al., 2023)
0.549
0,466
0.518
0.390
Setting 1
0.151
0.175
0.162
0.125
0.181
0.148
0.184
0.255
0.214
0.171
0.233
0.197
Setting
0.445
0.529
0.484
0.377
0.551
0.447
0.426
0.609
0.502
0.276
0.375
0.318
Setting 3
0.513
0.629
0.565
0.419
0.606
0.495
0.449
0.617
0.520
0.351
0.489
0.409
Setting 4
0.488
0.619
0,546
0,400
0.575
0.472
0.473
0.656
0.549
0.351
0.475
0,404
CONTRASTE-MTL
0.736
0.744
0.740
0.653
0.667
0.661
0.722
0.763
0.742
0.642
0.617
0.629
Table 9: Comparing ASTE results obtained using ChatGPT and CONTRASTE-MTTL All four settings are described in the text. Setting 3 gives our best ChatGPT results, however substantially outperformed by CONTRASTE-MTL.
A Appendix
tion. Here the prompt that we use is: Given the restaurant/laptop review: sent, find all the (as- pect, opinion, sentiment) triplcts appearing in the review in JSON format, with review and triplets as the keys. No explanations are needed and do not provide any text in the response.
A.1 Algorithm For Target Construction
Algorithm 1 Converting opinion triplets associated with a sentence into a linearized target sequence
Impul: Griplels: [asspect, opinion, sentiment), ...l List of opinion triplets for the given sentence. aever_ssrring-
Setting 2: Zero-shot with task definition. Here, we prepend the following ASTE task definition to the prompt mentioned above: Aspects are nouns or phrases appearing in the text that indicate spe- cific attributes of the entity being reviewed. Opin- ions are adjectives or phrases that express specific sentiments towards the aspects. Sentiments could be either positive, negative, neutral. Setting 3: Few-shot In-Context-Leannn (ICL) with the same exemplars. To design the prompt in this setting, after the task definition mentioned in Setting 2, and before the final in- struction defined Setting above, we add 100 randomly selected (sentence, target output) sam- ples from the corresponding training set. Please note that the same 100 samples are used part of the prompt for each test set sentence. Setting 4: Few-shot ICL with dynamically se- lected exemplars. Here, for each test set sen- tence, we use a different set of randomly selected 100 samples from the corresponding training set as part of the prompt. While no model parameters are updated in our experiments with ChatGPT, our goal in this setting is to investigate whether covering broader range of training set samples helps improve scores.
for triplet triplets do
finer_ssring += <aspect>
snnear_string += triplct(0)
linear_string += <opinion>
limear_string += triple([1]
linear_string += <sentiment>
linear_string += triple([2]
end for lineer_string += [SSEP]
linear_striie = lineer_strivve
A.2 Experiments With ChatGPT
Natural Language Processing, in recent times, has been revolutionized by the evolution of Large Lan- guage Models (LLMs) such as GPT-3 (Brown et al., 2020), PaLM (Chowdhery et al., 2022). Llama (Touvron et al., 2023), etc. ChatGPT (OpenAI, 2023b), powered by GPT-3.5 and GPT-4 (OpenAT, 2023a), has pioncered the excitement around LLMs by achieving remarkable zero-shot and few-shot in- context learning (ICL) (Brown et al., 2020) results for unseen tasks, without any parameter updates.
In this work, we carried out exhaustive experi- ments investigate how well can ChatGPT solve ABSA tasks. The ChatGPT version used by us is gpt-3.5-turbo. The temperature parameter was set to in order to avoid variations in ChatGPT- generated outputs. This will help us to reproduce the results in the future with the same settings. For generating the responses, the max_ joke_o parame- ter was set to 512. We experimented with 4 differ- ent prompts detailed below get the response for each test set sentence. We take the example of ASTE to explain our experimental settings: Setting 1: Full zero-shot with no task defini-
ASTE results obtained using ChatGPT are re- ported and compared against CONTRASTE-MILL in Table 9 above. First, we observe that across all 4 datasets, using the task definition helps to im- prove scores considerably. Prompting with training set examples further improves the scores. How- ever, we do not observe any advantage of Setting 4 over Setting 3. This is expected since the model does not remember the training set examples it

---

Res15
Res16
Rest-ACOR
Lap-ACOS
Model
P.
R.
P.
R.
F1
R.
F₁
P.
R.
CharGTT
0.257
0.271
0.262
0.367
0.407
0.386
0.396
0.386
0.391
0.253
0.224
0.237
Paraphrase
0.457
0,479
0.468
0,575
0.582
0.578
0.597
0.388
0.592
0.434
0.424
0.429
ACOS- Base
0.451
0.465
0.458
0.569
0.598
0.583
0.598
0.597
0.597
0.436
0.426
0.431
ACOS-Contra
0.471
0,484
0.478
0.587
0.610
0.598
0.608
0.602
0.605
0.450
0,441
0.446
Table 10: Comparative ACOS results on the Lap-ACOS, Rest-ACOS (Cai et al., 2021), and Res15, Res16 (Zhang et al., 202 datasets. ChatGPT results are obtained using 100-shot In Context Learning (ICL) prompts. The highest F1 scores each dataset are highlighted in bold. The second highest F1 scores are underlined.
was prompted with earlier while generating the re- sponse for a new test set sentence. Its memory is limited to the context of the prompt.
Model
Res15
Res16
ChatGPT
0.469
0.561
TAS-TS-WW-BIO-CRF
0.533
0.616
We also compare ChatGPT results with the ones reported in (Han et al., 2023) where the au- thors investigate the capability of ChatGPT in solv- ing different information extraction tasks, includ- ing ABSA. We observe that our Sctting 3/4 results comfortably outperform them. Finally, comparing our Setting 3 results with the ones obtained using our proposed CONTRASTE-MIT model, we ob- serve huge gap in performance. With necessary task-specific adjustments made to Setting 3, report the ChatGPT results for other ABSA tasks in tables 10, 11, and 12 respectively, and observe similar trends in performances. These observa- tions reinstate the findings of Han et al. (2023) and demonstrate that ChatGPT is not equipped enough to produce state-of-the-ae -frt results for ASBA tasks.
BART-Phnnse-Joo--TASSD
0.585
0.602
T5- DS-P
0.614
0.698
TT-Sentence-JJonn-TASS)
0.611
0.675
GAS-Extraction
0.615
0.694
PARAPHRASE
0.630
0.719
LEGO-ABSA
0.617
0.688
Seq2Path (k=8)
0.633
0.721
EHG-Para
0.628
0.721
TASD-Base
0.634
0.709
TASD-Contra
0.664
0.747
Table II: Test F TASD scores the benehmark Res15, and Res16 6 datasets (Wan et al., 2020). The highest F1 scores on each dataset are highlighted in bold. The second highest F1 scores underlined.
datasets, Res15, and Resl6 released by Wan et al. (2020), and report our results in Table 11.
A.3 Advantage of Pre- Pre-Trainin On ACOS
We perform our experiments on four benchmark ACOS datasets, Restaurant-ACOS, Lapiop-ACOS released by Cai et al. (2021), and Rest15, Rest16 released by the authors of PARAPHRASE (Zhang et al., 20)1a), and report our detailed results in Table 10. We had to make minor change in fine-tuning templates by adding the special token <category> and its associated value before <aspect>, The remaining fine-tuning settings are similar ASTE as discussed in Sec. 3.2.
Among the baselines, TAS (Wan et al., 2020) highlights that the prediction of sentiment polari- ties depends on both the target (category) as well as the aspect terms, and accordingly proposes a novel approach for parget-sspect-sentine joint detec- tion, specifically improving the methods that pre- dict sentiment polarities based only on explicit tar- gets. Joint-TASD (Chebolu et al., 2021) transforms ABSA into abstract summary-like conditional generation. GAS-Extraction (Zhang et al.. 1b) is predecessor of the PARAPHRASE, paper that generates triplets with simple linearized tuple tem- plates. PARAPHRASE (Zhang et al., 2021a) dis- cussed before proposes to model triplet generation as natural language paraphrases. LEGO-ABSA (Gao al., 2022) solves multiple ABSA tasks by controlling the task prompts. Seq2Path (Mao al., 2022) proposes a novel method where they gener- ate sentiment tuples as paths of a tree. EHG-Para (Ly et al., 2023) leverages a novel Efficient Hybrid
A.4 Ad Advantage of Pre- Fre-Trainng On TASD
Here, we consider the task of Target (category) Aspect Sentiment triplet Detection (TASD) (Wan et al., 2020). Given a sentence, for example, "the spicy tuna roll was unusually good", the task is extract the (category, aspect, sentiment) triplet ('food quality', 'spicy tuna roll', POS). Further, there could multiple such triplets in a sentence. We perform our experiments on two benchmark

---

Model
14Res
15Res
16Res
Lap14
tive subtask targets as sequences mixed with aspect term/opinion term word indices and sentiment class indexes. GAS-R (Zhang et al., 2021b) again tack- les ABSA tasks a unified generative framework by formulating targets using simple linearized tem- plates. EHG (Lv et al., 2023) leverages novel Effi- cient Hybrid Transformer to generate the semantic and position information of AESC targets in paral- lel. SentiPrompt (Li al., 2021 injects sentiment knowledge connecting aspects and opinion terms using sentiment knowledge-enhen enhanced prompts tunc the language model.
CharGPT
0.634
0.556
0.623
0.541
BART-ABSA
0.785
0.699
0.757
0.682
GAS-R
0.790
0.688
0.757
0.658
EHG
0.793
0.700
0.771
0.685
SentiPrompt 0.811
0.742
0.798
0.708
AESC-Base
0.818
0.729
0.781
0.701
AESC-Contra
0.826
0.741
0.810
0.731
Table 12: Comparative test AESC results on the benchmark datasets (Peng et al., 2020b). The highest and the second highest F1 scores on each dataset are respectively highlighted in bold and underline.
Similar to the previously discussed tasks, in Ta- ble 12, AESC-Base and AESC-Contra refer to our fine-tuned T5 model variants (with templates suit- ably updated for AESC) without and with con- trastive pre-training respectively. Consistent with our previous observations, here again we establish the advantages of our proposed pre-training strat- egy as AESC-Contra outperforms AESC-Base and our strongest competitor SentiPrompt with over- all 2.65%, and 1.65% F1 gains, averaged across the four datasets. For our experiments, our fine- tuning templates only consist of the <aspect> and <sentiment> placeholders followed by their spective values. All the above experiments fur- ther highlight the generalizability of our templates over PARAPHRASE. When performing our exper- iments for the respective tasks, i.c. ACOS, TASD, and AESC, we did not make any changes in the pre-training settings. The fine-tuning settings are similar to ASTE as discussed in Section 3.2.
Transformer and proposes novel global hybrid loss in combination with bipartite matching.
In Table 11, TASD-Base refers to the T5 encoder- decoder framework trained with our proposed fine-tuning templates (updated for TASD). TASD- Contra refers to the fine-tuning of TASD-Bass from checkpoint that is pre-trained using proposed strategy (refer Section 2.1). We ob- serve that TASD-Base performs slightly better on Res15 than the three strongest baselines, EHG- Para, PARAPHRASE, and Seq2Path. However, its performance on Res16 is weaker than the other three. Leveraging the advantages of supervised contrastive pre-training, TASD-Contra performs substantially better than TASD-Base while comfort- ably outperforming EHG-Para, PARAPHRASE, and Seq2Path with overall 4.67%, 4.65%, and 1.88% F, gains, averaged over the two datasets. For our experiments, here again we had to make a minor change our fine-tuning templates by adding the special token <category> and its asso- ciated value before <aspect>. Also, we remove the <opinion> placeholders from the targets.
A.5 Advantage of Pre- Tre-Training On AESC
Lastly, we consider the task of Aspect Extraction and Sentiment Classification (AESC). Here, given a sentence, for example, "the spicy tuna roll was unusually good", the task is extract the (aspect, sentiment) pair ('spicy tuna roll'. POS). Further, there could be multiple such pairs in given sen- tence. We perform our experiments on a total of four benchmark datasets (Peng et al., 2020b) from two domains; 14Res, 15Res, and 16Res belong- ing to the Restaurant domain, and Lap14 built on Laptop reviews, and report our results in Table 12. Among the baselines, Unified-BART-ABS, (Yan et al., 2021) converts all ABSA subtasks into a unified generative formulation by redefining respec-

---

Validation with beta (Lap14)
Validation (14Res)
0.51
0.70
0.22
0.03
0832
alpha
alpha
Validation with beta (15Res)
Validation
(16Res)
0.70
0.79
0.77
0.75
alpha
alpha
Choncing optimal acting
Validation F1 Scores with alpha 1.0 (Lap14)
Validation F1 Scores with alpha 0.2 (14Res)
0825
0831
0:72
beta
beta
Validation F1 Scores with alpha 0.8 (15Res)
Validation F1 Scores with alpha 0.8 (16Res)
0.78
0.75
0:2
beta
beta
Chocsing optinel giver opeimal
Figure 3: Task weight tuning the dev set for Opinion Term Detection (OTD) and Triplet Count Estimation (TCE). We first optimize for a (a), and then for 3 (b).


Reviews:
[{"review_id": "XqGn5O7k0Z", "reviewer": "Reviewer_ZRNq", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and effort in giving us your valuable feedback. We are happy to note that you have appreciated our detailed illustration of the proposed method and comprehensive experimental results. Please find below our clarifications/answers to your queries:  \n\n1. \"The motivation of the paper is unclear...\"\n\nWe understand your concern. Please note from line 45 that all existing works on ASTE have predominantly tried to develop newer, and often more complex fine-tuning techniques for the task. Our primary motivation for the work, as mentioned on line 48, is however to design an efficient pre-training strategy to enhance the aspect-aware sentiment understanding of our proposed model. This helps us to improve the downstream performances not only for ASTE but also for other ABSA tasks, such as ACOS, TASD, and AESC, as demonstrated in the paper. As detailed in Section 5, the novelty of our work further lies in the way we apply contrastive learning on aspect-level sentiment embeddings of masked opinion words (obtained using carefully-curated aspect-based prompts to guide the decoder) instead of sentence-level embeddings. Overall, what makes our approach different from existing works is that we propose a generic pre-training technique to improve multiple ABSA tasks simultaneously rather than focusing on more sophisticated task-specific fine-tuning techniques for respective ABSA tasks. We shall clarify these points more clearly in the camera-ready draft (if accepted).\n\n2. \"The mechanism of OTD is unclear...\"\n\nThank you for highlighting your concern, however we would beg to differ on the following two grounds. First, in the reference paper (https://aclanthology.org/2022.findings-acl.156.pdf), which motivated us to include the OTD module, please note from Section 3.1 (in the referred paper) that the main task of “Autoregressive Entity Linking” is formulated as a “language generation task”, whereas the auxiliary subtask of “Entity Mention Detection” (refer Section 3.2) is modeled as a token-wise binary classification task. Similarly we formulate ASTE, our main task, as a sequence-to-sequence generative task, whereas the auxiliary subtask of OTD is formulated as a sequence-tagging task using the BIO scheme, thereby making it a token-level 3-way classification task as described on line 274.\n\nSecond, as mentioned on line 266, we hypothesize that OTD will help to better detect the opinion span boundaries which in turn should help to improve ASTE (mainly in opinion term extraction and sentiment prediction). As already acknowledged by you, in Table 6 (ablation results), we report that the inclusion of OTD module helps to improve the Opinion-F1 scores by 2.4% (0.848 vs. 0.828), Sentiment-Accuracy scores by 1.8% (0.784 vs. 0.770), and finally the ASTE/Triplet-F1 scores by 1% (0.692 vs. 0.685). Also, please note from line 301 that this mutual influence results from the fact that we are training our model (CONTRASTE-MTL) in a multi-task setup, jointly optimizing the generative, OTD, and TCE loss functions.\n\n3. \"The paper lacks a deeper analysis into the TCE module...\". We sequentially provide answers to the three concerns raised by you:\n\n- 3.1. \"It is a regressor instead of classifier...\"\n\nFirst of all, we would like to clarify that the number of triplets varies across sentences. Also, a test set sentence may well contain more triplets than any other sentence it has seen during training. Hence, we modeled the auxiliary subtask of triplet count estimation (TCE) as a regressor instead of a classifier since we do not know the exact no. of triplets apriori. Please note from line 287, that TCE is introduced to implicitly guide the decoder when to stop. As mentioned before, this implicit guidance is expected through the joint optimization of loss functions for the three modules, encoder-decoder, OTD and TCE. Please note that there is no explicit connection between the output of the TCE module and the decoder. As mentioned on line 283, the explicit guidance for the decoder on how many triplets to generate is provided through supervision from ground truth target sequences.\n\nWe agree that the TCE module can produce float numbers. During training, we drive the model behavior by minimizing the Mean Squared Error loss as mentioned on line 292. During inference, we round off the output to the nearest integer (and 2.5 to 2). Therefore, for a TCE-generated value of 2.4, we would ideally want the decoder to stop after generating 2 triplets. However, please note again that there is no such direct influence. We also do not post-process the decoder-generated sequence to match the same number of triplets as predicted by the TCE module before calculating the reported results.\n\n- 3.2. \"How much chance it can predict the correct aspect term number during inference time?\"\n\nPlease note that the motivation behind the inclusion of the TCE module was to improve the model’s capability to predict the **correct no. of triplets** for a given sentence, and **not the correct no. of aspect terms**. The same aspect may be repeated across several triplets. For example, the sentence “the food was absolutely fresh and delicious, however overpriced” contains three opinion triplets as follows: (food, absolutely fresh, positive), (food, delicious, positive), and (food, overpriced, negative). Here, all three triplets contain the same aspect term “food”. The inclusion of the TCE module however improves, not only the overall ASTE performance as already acknowledged by you, but also the Aspect-F1 scores by 1% (0.840 vs. 0.832) (referring to the ablation results reported in Table 6). We however attribute this observation to the improved capacity of the model to generate the correct no. of triplets, which in turn directly impacts all the scores.\n\nFollowing your suggestions, we tried to quantify the performance of the TCE module separately and also to what extent it matches with the actual no. of triplets generated by the decoder. Please note that for calculating the results, we round off the TCE-generated float values to the nearest integers, as mentioned above. The results are reported in the table below:\n\n| Dataset                                                                                          | 14Res | 15Res | 16Res | Lap14 |\n|----------------------------------------------------------------------------------------------------------------------|-------|-------|-------|-------|\n| TCE Accuracy                                                                                                         | 92\\%  | 88\\%  | 90\\%  | 85\\%  |\n| Percentage of times the no. of triplets predicted by TCE matches the actual no. of triplets generated by the decoder | 93\\%  | 93\\%  | 95\\%  | 92\\%  |\n|\n\nWe observe that the TCE module is quite accurate, predicting the correct no. of triplets close to 89% of the times, averaged across all 4 ASTE datasets. Further, in more than 93% of the cases on average, the no. of triplets predicted by the TCE module matches the actual no. of triplets generated by the decoder, thereby demonstrating the synergy between the two components.\n\n\n- 3.3. \"If training with a randomly-generated pesudo labels..\"\n\nWe observe from above that the TCE module is fairly accurate, and that the TCE and decoder predict the same no. of triplets in more than 93% of the cases. If the TCE module is trained with randomly-generated pseudo labels, we expect a negative impact on the overall ASTE scores, despite accepting the fact that the TCE and decoder do not have any explicit influence apart from their losses being optimized together.  In the best scenario, the model can learn to ignore this random signal, but then the overall ASTE performance would be at best as reported without the TCE module (second last row in Table 6). \n\nWe are happy to address your concerns/suggestions, and would like to answer any follow-up questions that you may have. However, please note that our main contribution is to propose a novel pre-training strategy that would improve multiple ABSA tasks rather than focusing on a specific ABSA task. From Table 5, we observe that ASTE-Base has a triplet F1 score of 0.671 averaged across all 4 datasets, which improves to 0.680 (CONTRASTE-Base) with our proposed contrastive learning-based pre-training technique. Training the model in a multi-task setup (CONTRASTE-MTL), with OTD and TCE modules included, helps us to outperform Span-BiDir, our strongest baseline, by a margin of 1% in triplet F1 scores (0.692 vs. 0.685). Finally, from Tables 8, 9, 10, and 11, we find that our proposed strategy to pre-train an encoder-decoder network by means of applying contrastive learning on aspect-level sentiment embeddings improves all other ASBA tasks. \n\nWe pledge to report the suggested clarifications and results in the camera-ready draft (if accepted). In this regard, we sincerely request you to kindly reconsider your scores, if you find our explanations/clarifications satisfactory.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Detailed illustration in both text and picture about the proposed method, which I can follow easily. \n2. Comprehensive and promising experiment results.", "reasons_to_reject": "1. The motivation of the paper is unclear or not strong. 'different from existing work' could not be a good motivation.\n\n2. The mechnism of OTD is unclear. OTD is a token-level task while the generative ASTE is formulated as a sequence-to-sequence generation one, so basically they should not have an mutual influence. \nBesides, I also note that in the reference paper as your motivation for add OTD, the target sequence of the two tasks are the same, but it is not the case in this paper's setting, which makes readers more doubtful about the effectiveness of OTD module (although in your ablative results it really works a bit).\n\n3. The paper lacks a deeper analysis into the TCE module, though ablation results show it benefits the model's performance. I am curious about how it really works. My points are as follows:\n1) It is a regressor instead of classifier, meaning it could produce float numbers. In this case, how the model could treat a float number output (e.g., 2.4, the decoder intends to stop after producing 2 or 3 aspect term) at inference time?\n2) How much chance it can predict the correct aspect term number during inference time? \n3) If training with a randomly-generated pesudo labels regardless of its correctness can still enhance the performance.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes CONTRASTE-MTL, a novel pre-training strategy using contrastive learning to improve the performance on ASTE task.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "k7htyjkb02", "reviewer": "Reviewer_8sQy", "summary": "", "questions": "1. As mentioned in the paper, authors respectively fine-tune the pre-trained model weights for the downstream ASTE task using AdamW optimizer with a learning rate of 1e-4 for 14Res and 16Res, and 3e-4 for 15Res and Lap14. Why the learning rates are different? Do I have to readjust the learning rate if I use a different dataset?\n2. As mentioned in the paper, authors start by optimizing α with β set to 0.4, then optimize β given the optimal α values. Why do authors determine the values of α and β in this way? The reason for this is not explained in the paper.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and effort in giving us your valuable feedback. We are extremely happy to note that you have generously appreciated the novelty of our proposed pre-training strategy, and its utility in achieving state-of-the-art results for multiple ABSA tasks. Thank you once again for appreciating the detailed narrative of our proposed method, the universality of the generic placeholder-based templates, and the reliability, rigor, and explanation of our experimental results, among other things. Please find below our clarifications/answers to your queries:\n\n1. \"The related works are not well organized.\"\n\nWe note that you have acknowledged the adequacy of the related works as one of the strengths, however we understand your concern. We accept that we could not organize the entire related works section within the main draft owing to space constraints. As already pointed out by you, the novelty of our proposed approach primarily lies in how we apply supervised contrastive pre-training with carefully-curated aspect-based prompts in order to improve multiple ABSA tasks. Accordingly, we organized the prior studies on the application of supervised contrastive learning for ABSA within the main draft. The ASTE baselines are detailed in Section A.3 in the appendix. Similarly, the baselines for the tasks of ACOS, TASD, and AESC are respectively detailed in appendix sections A.4, A.5, and A.6. We pledge to organize the related works section better in the camera-ready draft by utilizing the extra page we will get.\n\n2. \"The paper lacks a comparative analysis with ChatGPT.\"\n\nThank you for this suggestion. We carried out exhaustive experiments to understand the capability of ChatGPT in solving multiple ABSA tasks. The ChatGPT version used by us is gpt-3.5-turbo. The “temperature” parameter was set to 0 in order to avoid variations in ChatGPT-generated outputs which is a known fact. This further helps us to reproduce the results in the future with the same settings. For generating the responses, the “max_tokens” parameter was set to 512. We experimented with 4 different prompts to get the response for each test set sentence. We take the example of ASTE to explain our experimental settings:\n\n- Setting 1: Full Zero-shot with no task definition. Here the prompt that we use is:\n\nGiven the restaurant/laptop review: {sent}, find all the (aspect, opinion, sentiment) triplets appearing in the review in json format, with review and triplets as the keys. No explanations needed and do not provide any text in the response.\n\n- Setting 2: Zero-shot with task definition. Here, we simply prepend the following task definition for ASTE to the prompt mentioned above: \n\nAspects are nouns or phrases appearing in the text that indicate specific attributes of the entity being reviewed. Opinion terms are  adjectives or phrases that express specific sentiments towards the aspects. Sentiments could be either positive, negative or neutral.\n\n- Setting 3: Few-shot In-Context-Learning (ICL) with the same training set samples.\n\nTo design the prompt in this setting, after the task definition mentioned in Setting 2, and before the final instruction mentioned in Setting 1, we add 100 randomly selected (Sentence, Target Output) samples from the corresponding training set. Please note that the same 100 samples are used as part of the prompt for each test set sentence. \n\n- Setting 4: Few-shot ICL with dynamically selected training set samples:\n\nHere, for each test set sentence, we use a different set of randomly selected 100 samples from the corresponding training set as part of the prompt. While no model parameters are updated in our experiments with ChatGPT, our goal in this setting is to see whether covering a broader range of training set samples helps to improve scores.\n\nFollowing are the ASTE results with ChatGPT on the benchmark ASTE-Data-V2 datasets:\n\n| Setting       | 14Res |  |  | 15Res |  |  | 16Res |  |  | Lap14 |  |  |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|    | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 |\n| 1 | 0.151 | 0.175 | 0.162 | 0.125 | 0.181 | 0.148 | 0.184 | 0.255 | 0.214 | 0.171 | 0.233 | 0.197 |\n| 2 | 0.445 | 0.529 | 0.484 | 0.377 | 0.551 | 0.447 | 0.426 | 0.609 | 0.502 | 0.276 | 0.375 | 0.318 |\n| 3 | 0.513 | 0.629 | 0.565 | 0.419 | 0.606 | 0.495 | 0.449 | 0.617 | 0.520 | 0.351 | 0.489 | 0.409 |\n| 4 | 0.488 | 0.619 | 0.546 | 0.400 | 0.575 | 0.472 | 0.473 | 0.656 | 0.549 | 0.351 | 0.475 | 0.404 |\n| ChatGPT 5-shot ICL results for ASTE reported in https://arxiv.org/pdf/2305.14450.pdf  | - | - | 0.549 | - | - | 0.466 | - | - | 0.518 | - | - | 0.390 |\n| Our best-reported results with CONTRASTE-MTL | 0.736 | 0.744 | 0.740 | 0.653 | 0.667 | 0.661 | 0.722 | 0.763 | 0.742 | 0.642 | 0.617 | 0.629 |\n|\n\n\nWe observe that across all 4 datasets, using the task definition helps to improve scores considerably. Prompting with training set examples further improves the scores. However, averaged across all 4 datasets, we do not observe any advantage of Setting 4 over Setting 3. This is expected since the model does not remember the training set examples it was prompted with earlier while generating the response for a new test set sentence. Its memory is limited to the context of the prompt.\n\nInterestingly, an arxiv paper https://arxiv.org/pdf/2305.14450.pdf also reports results for different information extraction tasks using ChatGPT, including ABSA. Please refer to Table 1 of this arxiv paper. For your convenience, we have reported their ASTE results in the table above. We observe that our Setting 3/4 results comfortably outperforms them. Finally, we also compare the ChatGPT results (Setting 3) with our results with CONTRASTE-MTL reported in the paper, and find a huge gap in performance. This confirms the findings of the arxiv paper, and demonstrates that ChatGPT is not equipped enough to give us state-of-the-art results for ASBA tasks.\n\nWith necessary task-specific adjustments made to Setting 3, here are the ChatGPT results (F1-scores) for the other ABSA tasks reported in the paper (in comparison to our model scores):\nPlease note that the results are obtained on the test sets of respective task-specific datasets.\n\n| Model        | 14Res | 15Res | 16Res | Lap14 | Rest-ACOS | Laptop-ACOS |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|\n| ACOS-ChatGPT | -     |  0.262 | 0.386 | -     |    0.391   | 0.237       |\n| ACOS-Contra  | -     | 0.478 | 0.598 | -     | 0.605     | 0.446       |\n| TASD-ChatGPT | -     | 0.469 | 0.561 | -     | -         | -           |\n| TASD-Contra  | -     | 0.664 | 0.747 | -     | -         | -           |\n| AESC-ChatGPT | 0.634 | 0.556 | 0.623 | 0.541 | -         | -           |\n| AESC-Contra  | 0.826 | 0.741 | 0.810 | 0.731 | -         | -           |\n|\n\nFinally, we pledge to include these results in the camera-ready draft.\n\n3. \"Pre-training a large language model is extremely resource-consuming...\"\n\nWe understand your concern. However, we would like to clarify a few things. First, please note that we are not pre-training the T5 model from scratch which is an extremely resource and time-consuming task. We start from the pre-trained T5 checkpoint downloaded from the Huggingface library and continue pre-training it using our proposed strategy to improve its aspect-level sentiment understanding before we start fine-tuning the model for respective ABSA tasks. In essence, our proposed method can also be viewed as a two-step fine-tuning process, where the first step is a one-time process where we train the model using contrastive learning, and the second step is to take the updated model weights and further fine-tune them for respective ABSA tasks. Second, please note that all our experiments are performed with T5-base which is a relatively smaller T5 variant with 220M parameters (T5-XXL as 11B parameters). Finally, for the first step, we do not use any external data for performing contrastive pre-training. We limit our experiments to the data obtained from the training sets of our task-specific ABSA datasets, as described in the Section 3.2, thereby avoiding any additional overhead. Hence, we believe our proposed method can be efficiently applied to other situations.\n\n4. \"Pre-training large language models may not be sufficient..\"\n\nTrue. However, we would like to consider this as an advantage of our proposed pre-training technique. As detailed in Section 5 (line 565), prior works on applying supervised contrastive learning for ABSA usually rely on external large-scale sentiment-annotated corpora to learn sentiment knowledge. We, on the other hand, efficiently pre-train our model by only effectively leveraging the limited data from the training sets of benchmark ABSA datasets. We believe that we can further improve our reported results if we make use of additional resources for large-scale pre-training. We shall include these clarifications in the camera-ready draft.\n\n5. Query regarding different learning rates for different datasets.\n\nYes. Please note from Table 3 that all 4 ASTE benchmark datasets have different no. of data points, and different distributions of positive, negative, and neutral sentiment triplets. Further, each of them have their own pre-defined train-validation-test splits. While fine-tuning the contrastive pre-trained T5 model weights on the respective datasets, we tuned our hyperparameters on the corresponding validation sets. Please note that hyperparameter tuning on a held-out validation/development set is a standard procedure used across papers. We reported the hyperparameters that helped us to obtain the best triplet-F1 scores on the respective validation sets. In our case, the best learning rates were different for different datasets as reported in Section 3.2. Finally, it may not be always required to readjust the learning rate unless you take a dataset with very different characteristics to work on.\n\n6. Query regarding setting the α with β parameters.\n\nPlease refer to line 338 under Section 3.2 where we have referred to the paper which motivated us to include the OTD module and train our final model, CONTRASTE-MTL, in a multi-task setup. Similar to our case, the referred paper also trains their final model by optimizing a joint loss involving the loss functions for the main task and two auxiliary tasks. We take a strategy similar to the one used in the referred paper to select the auxiliary task weights. We start by optimizing α with β set to 0.4. We then optimize β given the optimal α values. During our experiments, we verified that reversing the order did not change the optimal values. Another strategy we could have used is a grid-search to choose the optimal alpha and beta values simultaneously, however it is a computationally expensive process. We assure you to include a detailed explanation of our parameter-tuning strategies in the camera-ready draft.\n\nOverall, we are happy to address your concerns/suggestions, and would like to answer any follow-up questions that you may have. Also, we would sincerely request you to kindly reconsider your scores, if you are satisfied with our explanations and additional results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper proposes CONTRASTE, a novel pre-training strategy using contrastive learning to improve the performance of the downstream ASTE task. The extensive experiments and detailed analysis also support the proposed method. This work is beneficial to the researchers who study the ASTE task, the ABSA tasks, supervised contrastive learning, etc. Please also refer to the “strengths” for more details.", "reasons_to_reject": "The paper lacks a comparative analysis with ChatGPT. Additionally, pre-training large language models may not be sufficient, because the amount of training data is not enough. Please also refer to the “weaknesses” for more details.", "questions_to_authors": "1. As mentioned in the paper, authors respectively fine-tune the pre-trained model weights for the downstream ASTE task using AdamW optimizer with a learning rate of 1e-4 for 14Res and 16Res, and 3e-4 for 15Res and Lap14. Why the learning rates are different? Do I have to readjust the learning rate if I use a different dataset?\n2. As mentioned in the paper, authors start by optimizing α with β set to 0.4, then optimize β given the optimal α values. Why do authors determine the values of α and β in this way? The reason for this is not explained in the paper.", "paper_topic_and_main_contributions": "This paper proposes CONTRASTE, a novel pre-training strategy using contrastive learning to improve the performance of the downstream ASTE task. First, the paper derives aspect-based prompts with corresponding sentiments masked. Then, supervised contrastive learning (SCL) is applied on the decoder-generated aspect-level sentiment embeddings of the masked tokens. The paper demonstrates that the proposed pre-training strategy results in a better downstream performance for ABSA tasks such as ASTE, ACOS, TASD, and AESC, than performing SCL on sentence-level sentiment embeddings. After that, the paper proposes generic placeholder-based templates to fine-tune the encoder-decoder framework. Finally, the paper presents a multi-task model, CONTRASTE-MTL, that achieves SOTA results on the majority of ASTE benchmark datasets. The results demonstrate that the proposed pre-training scheme can improve the performance of other ABSA tasks such as ACOS, TASD, and AESC.\n\n-strengths\n1. The problem is cutting edge. This paper studies supervised contrastive pre-training with aspect-based prompts for aspect sentiment triplet extraction, which has been less explored.\n2. The supervised contrastive pre-training with aspect-based prompts for aspect sentiment triplet extraction is somewhat novel and the method is well-designed.\n3. The narrative and logic of the paper are very clear.\n4. Figures are clear, and the design of these figures is nice.\n5. The generic placeholder-based templates have greater universality.\n6. The proposed pre-training strategy can achieve state-of-the-art results for other ABSA tasks.\n7. The experiments are extensive and rigorous, and the explanation of the experiments is detailed.\n8. The related works are adequate.\n9. The experimental results have high reliability.\n10. The paper is well-written and structured.\n11. The limitations are frank.\n\n-weaknesses\n1. The related works are not well organized.\n2. The paper lacks a comparative analysis with ChatGPT.\n3. Pre-training a large language model is extremely resource-consuming, and the method proposed in this paper cannot be easily, quickly, and cheaply applied to other situations.\n4. Pre-training large language models may not be sufficient, because the amount of training data is not enough.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZA58uZIoCQ", "reviewer": "Reviewer_RHnr", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and effort in giving us your valuable feedback. We are extremely happy to note that you have generously appreciated our coherent presentation, the motivation and novelty of our proposed pre-training strategy, and its utility in achieving state-of-the-art results for multiple ABSA tasks, the justification behind the inclusion of OTD and TCE modules for fine-tuning, and finally the comprehensiveness of our experiments. Please find below our clarifications/answers to your queries:\n\n1. \"...recommended to include additional novel approaches.\"\n\nFrom Table 5, please note that we have compared our ASTE results with a total of 16 different baseline approaches, which, to the best of our knowledge, is the most by any published paper on the task. Further, in order to demonstrate the novelty of our proposed approach, and the importance of our design decisions, we report the scores for 3 different variants of our model, ASTE-Base, CONTRASTE-Base, and CONTRASTE-MTL. As per the suggestions of other reviewers, we have also performed detailed experiments with chatGPT, which we plan to include in the final version. We try different zero-shot and few-shot prompts to find out that CONTRASTE-MTL results are substantially better than the best results obtained using ChatGPT. We observe the same trend for all other ABSA tasks. We therefore conclude that ChatGPT is not equipped enough to give us state-of-the-art results for ASBA tasks. Here, we would request you to kindly provide us with some references for papers which we might have missed unknowingly. We assure you to include them as part of our camera-ready draft.\n\n2. \"The description of the TCE module in the paper's methodology is concise..\"\n\nThank you for pointing this out. As mentioned on line 288, the Triplet Count Estimator (TCE) is a simple regressor built on top of the encoder using fully connected networks (FCN). More specifically, we take the mean encoder embeddings of the input sentence (dim 768) and pass it to the first FCN (dim 128 X 768). Next, we pass the outputs of this layer to the second FCN (dim 1 X 128) that brings it down to a float value. The output of the layer is passed into the MSE loss function along with the ground truth no. of triplets. As mentioned on line 301, the MSE loss is jointly minimized with a weighing term along with the other 2 objectives. \n\nAlso, please note that the number of triplets varies across sentences. Also, a test set sentence may well contain more triplets than any other sentence it has seen during training. Hence, we modeled the auxiliary subtask of triplet count estimation (TCE) as a regressor instead of a classifier since we do not know the exact no. of triplets apriori. Please note from line 287, that TCE is introduced to implicitly guide the decoder when to stop. As mentioned before, this implicit guidance is expected through the joint optimization of loss functions for the three modules, encoder-decoder, OTD and TCE. Please note that there is no explicit connection between the output of the TCE module and the decoder. As mentioned on line 283, the explicit guidance for the decoder on how many triplets to generate is provided through supervision from ground truth target sequences.\n\nWe assure you to include these details in the camera-ready draft. Also, we would be more than happy to answer any follow-up questions that you may have.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1、The authors provide a clear statement of the research problem, and the overall content demonstrates a relatively coherent logic.\n2、The methods employed in the paper, including the reasonable adoption of pre-training techniques and the incorporation of OTD and TCE modules during fine-tuning, are well-justified. Additionally, the explanations of the methods are presented in a clear manner.\n3、The authors conducted extensive and comprehensive experiments to support their methods, and they also performed supplementary experiments on other ABSA tasks.", "reasons_to_reject": "1、\tIn the comparative experiments with the relevant methods and Baseline, it is recommended to include additional novel approaches.\n2、\tThe description of the TCE module in the paper's methodology is concise, while further elaboration is required on the specific procedural steps for utilizing the TCE module to guide the generation of triplet quantities by the model.", "questions_to_authors": "", "paper_topic_and_main_contributions": "1、In this work, the authors focus on Aspect Sentiment Triplet Extraction (ASTE), which is a subtask of Aspect-Based Sentiment Analysis (ABSA). The goal of ASTE is to extract complete aspect-sentiment triplets from the text. To address the ASTE task, the authors propose a contrastive learning-based pretraining strategy and adopt a multi-task approach for model fine-tuning.\n2、In details, during the pretraining phase, the authors employ contrastive learning to adjust the aspect-level sentiment embeddings through distance metrics. In the fine-tuning phase, the authors introduce new prompt templates and incorporate the OTD and TCE modules for joint training. Moreover, the model is extended to multiple other tasks such as ACOS, TASD, and AESC.\n3、Overall, the authors have conducted comprehensive experiments based on ASTE and other ABSA tasks, and the overall logical structure of the paper is clear. However, the description of the TCE module appears to be relatively concise.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new pre-training strategy using contrastive learning for Aspect Sentiment Triplet Extraction. The experimental setup is sound and thorough. The proposed method is novel and provides good performance gains on the task. The authors should clarify a bit better the efficacy of individual components such as TCE and improve the related work section as well as the motivation behind their approach.
Decision: Accept-Findings