Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction (2024)

Abstract: Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review, which is the most representative and challenging task in aspect-based sentiment analysis. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. To tackle this issue, we propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels, aiming to filter out mismatches and thereby enhance the effectiveness of selftraining. We highlight two critical aspects to ensure the scorer’s effectiveness and reliability: the quality of the training dataset and its model architecture. To this end, we create a humanannotated comparison dataset and train a generative model on it using ranking-based objectives. Extensive experiments on public ASQP datasets reveal that using our scorer can greatly and consistently improve the effectiveness of self-training. Moreover, we explore the possibility of replacing humans with large language models for comparison dataset annotation, and experiments demonstrate its feasibility.1

heading: 1 Introduction
text: Aspect-Based Sentiment Analysis (ABSA) aims to recognize aspect-level opinions and sentiments from user-generated content (Pontiki et al., 2014). This problem has consistently attracted interest owing to its proficiency in distilling and summarizing fine-grained opinions from vast data (Do et al., 2019; Nazir et al., 2022; Zhang et al., 2023a). The most representative and challenging task in ABSA is Aspect Sentiment Quad Prediction (ASQP) (Cai
∗ The three authors contribute equally to this work. † Corresponding Authors
1We release our code and data at https://github. com/HITSZ-HLT/ST-w-Scorer-ABSA.
et al., 2021; Zhang et al., 2021a). This task formulates aspect-level opinions and sentiments as quadruples, each consisting of an aspect term, aspect category, opinion term, and sentiment polarity. For example, given a review “the food is great and reasonably priced,” the output of ASQP would be {(food, food_quality, great, positive), (food, food _prices, reasonably priced, positive)}.
As a fine-grained problem, ABSA faces the challenge of insufficient labeled data, which is particularly severe in the ASQP task. This issue limits the performance of existing models. Many efforts explore data augmentation methods to alleviate this issue. They synthesize new samples by modifying existing ones (Li et al., 2020; Hsu et al., 2021), applying self-training techniques (Wang et al., 2021), or utilizing generative methods (Yu et al., 2023; Deng et al., 2023; Zhang et al., 2023b). However, a significant limitation of these methods is that the synthetic samples often inevitably contain mismatches between sentences and labels, which can adversely affect model learning.
To reduce such mismatches, this paper introduces a pseudo-label scorer for data augmentation. As illustrated in Figure 1, the scorer assesses the degree of match between the review and its pseudolabel. If we have a sufficiently robust scorer, we can filter out all mismatched samples, thereby sig-
11862
nificantly enhancing the effectiveness of data augmentation. We propose that the effectiveness and reliability of this scorer hinge on two critical aspects: (1) the quality of the training dataset and (2) its architecture along with the training objective. We discuss these two aspects below.
For the first aspect, previous works typically produce negative labels by modifying real labels using heuristic rules (Wang et al., 2021; Mao et al., 2022). However, such negative labels are usually simplistic and patterned, limiting the scorer’s learning. To overcome this limitation, we create a humanannotated comparison dataset. Specifically, we train an ASQP model with existing labeled data, use it to infer several pseudo-labels for unlabeled data, and then have human annotators choose the most appropriate pseudo-labels. The labels chosen by annotators are designated as positive labels, while the rest as negative labels. Our dataset, in contrast to the rule-based datasets, is more challenging and better aligned with human judgment.
For the second aspect, previous works formalize label-scoring as a question-answering problem (Wang et al., 2021) or embed the discriminative matching token into the label (Mao et al., 2022). However, our findings suggest that these methods underperform in complex tasks like ASQP, due to their limited capacity to model the interactions between reviews and pseudo-labels. Recent works in preference optimization reveal that the language model itself can serve as a scorer (Rafailov et al., 2023; Yuan et al., 2023). This motivates us to use the conditional likelihoods that a generative model assigns to a pseudo-label as the measure of its quality. Compared with the previous methods, this approach enables the scorer to examine the plausibility of a pseudo-label in a token-by-token fashion, thus offering a more comprehensive and effective scoring. We then fine-tune this scorer on our comparison dataset using ranking-based objectives.
Upon developing this pseudo-label scorer, we apply it in a data augmentation framework, specifically opting for the self-training framework due to its simplicity. We conduct extensive experiments on public ASQP datasets to examine its effectiveness and further investigate the following questions: (1) how does the pseudo-label scorer perform using our comparison data and model architecture?; (2) is it feasible to replace humans with large language models to annotate the comparison data?; and (3) how to utilize the scorer to filter out low-
quality samples? Furthermore, inspired by Ma et al. (2023), we extend the application of this scorer, employing it as a reranker for multiple candidate labels, and assess its impact and effectiveness.
Our contributions are summarized as follows:
(1) To the best of our knowledge, we are the first to apply a pseudo-label scorer to data augmentation in the ASQP task.
(2) We investigate how to enhance the scorer’s effectiveness and reliability from both dataset and model architecture perspectives.
(3) We empirically demonstrate that the proposed pseudo-label scorer can significantly and consistently enhance the performance of existing models.

heading: 2 Background
text: Task Evolution. Aspect-Based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis problem, aiming at recognizing user opinions and sentiments towards specific aspects (Zhang et al., 2023a). Within ABSA, aspects are generally defined as aspect categories or aspect terms. Aspect categories are predefined categories like food_quality and service_general in restaurant reviews, or laptop_portability and display_quality in laptop reviews. Aspect terms are explicit mentions of these aspect categories in the text. Based on this, Pontiki et al. (2014) formally define four subtasks: aspect category detection, aspect category sentiment classification, aspect term extraction, and aspect term sentiment classification. These tasks sequentially identify aspects and determine their corresponding sentiments.
Opinion term refers to words or phrases that express subjective sentiments. Sentiment expressions towards aspects often rely on these opinion terms, making them vital clues for recognizing aspects and their sentiment polarities. Consequently, many researchers focus on aspect and opinion coextraction (Wang et al., 2016; Li and Lam, 2017; Fan et al., 2019; Chen et al., 2020; Zhao et al., 2020). Building on this, Peng et al. (2020) propose the Aspect Sentiment Triplet Extraction (ASTE) task, conceptualizing an aspect-level sentiment as a triplet consisting of an aspect term, opinion term, and sentiment polarity. Following ASTE, Cai et al. (2021); Zhang et al. (2021a) extend the triplet by incorporating the aspect category, evolving it into Aspect Sentiment Quad Prediction (ASQP). ASQP
is currently the most representative and challenging task within ABSA.
ABSA Methods. Early efforts in ABSA primarily focus on identifying one or two sentiment elements. They mainly design specific model structures to establish interactions between sentiment elements and their contexts, as well as among different sentiment elements (Wang et al., 2016; Li and Lam, 2017; Ma et al., 2017; Chen et al., 2017; Xu et al., 2018; Xue and Li, 2018; Li et al., 2018; Fan et al., 2019; Zeng et al., 2019). Recent efforts have shifted towards compound tasks like ASTE and ASQP, proposing various end-to-end methods. These includes machine reading comprehensionbased methods (Chen et al., 2021; Mao et al., 2021), table-filling methods (Wu et al., 2020; Chen et al., 2022; Zhang et al., 2022), span-based methods (Xu et al., 2021; Li et al., 2022; Cai et al., 2021), and generative methods (Yan et al., 2021; Zhang et al., 2021b,a; Mao et al., 2022; Gao et al., 2022; Hu et al., 2023a,b; Gou et al., 2023). Among these, generative methods have emerged as mainstream due to their universality and capacity to exploit rich label semantics.
Generative Aspect-Based Sentiment Analysis, abbreviated as GAS, is a unified generative framework proposed by Zhang et al. (2021b). The core idea of this framework is to transform sentiment elements into a label sequence and then use a SEQ2SEQ model to learn the dependencies between the input text and the label sequence. For the ASQP task, we can convert it into a label sequence using the following template:
seqlabel = c1 | s1 | a1 | o1 ; · · · ; cn | sn | an | on,
where ci denotes the aspect category, si denotes the sentiment polarity, ai denotes the aspect term, and oi denotes the opinion term.

heading: 3 Comparison Dataset
text: We need to construct a comparison dataset to facilitate the training and evaluation of the pseudo-label scorer. This dataset comprises samples each containing a review sentence accompanied by several pseudo-labels, where one is the positive label and the others are negative labels. We train the scorer by requiring it to assign high scores to positive labels and low scores to negative labels.
Previous works typically produce negative labels using heuristic rules (Wang et al., 2021; Mao et al.,
2022). They randomly modify elements in the existing positive labels, such as altering boundaries or conducting substitutions. Such negative labels are patterned and easily distinguishable, limiting the learning potential of the scorer. Therefore, this paper employs human annotators to construct this comparison dataset.

heading: 3.1 Data Preparation
text: Aligned with existing ASQP datasets (Cai et al., 2021; Zhang et al., 2021a), we collect reviews from two domains: Restaurant and Laptop. The restaurant reviews are from the Yelp Dataset2, and the laptop reviews are from the Amazon Laptop Dataset3 (Ni et al., 2019). We segment these reviews into individual sentences. Next, we employ the existing labeled dataset to train an ASQP model (Zhang et al., 2021b) and then utilize this model to generate four pseudo-labels for each review sentence via beam search.

heading: 3.2 Annotation Process
text: For a review sentence and its four pseudo-labels, annotators are presented with six options. The first four options correspond to the four pseudolabels, the fifth option indicates that none of the pseudo-labels are appropriate, and the sixth option suggests that the review sentence does not express any sentiment or the expressed sentiment is difficult to infer. When annotators choose the fifth option, they are required to write an alternative label.
The annotation process is organized into multiple batches, each containing about 200 samples. To ensure accuracy, every sample is independently annotated by three different annotators. In cases of discrepancy among their annotations, a fourth annotator steps in to resolve the inconsistency. Furthermore, at the conclusion of each batch, the four annotators meet to discuss and reconcile any disagreements. Each annotator is provided with annotation guidelines4 and existing labeled ASQP datasets. In instances where conflicts arise between the two, we prioritize adherence to the guidelines.
AI Annotation. Choosing the most appropriate label, while much simpler than annotating an ASQP
2https://www.yelp.com/dataset 3https://nijianmo.github.io/amazon/
index.html 4https://alt.qcri.org/semeval2014/ task4/data/uploads/semeval14_absa_ annotationguidelines.pdf and https://alt. qcri.org/semeval2016/task5/data/uploads/ absa2016_annotationguidelines.pdf.
label from scratch, remains a laborious task. Therefore, we explore the feasibility of using ChatGPT as a substitute for human annotators. To ensure the quality of AI annotations, we carefully craft prompts for each ASQP dataset. Additionally, we incorporate three strategies to enhance the annotation process: self-consistency, self-assessment, and rationale augmentation. Details of AI annotation can be found in Appendix A.

heading: 3.3 Statistics
text: We construct two human-annotated and four AIannotated comparison datasets. Their basic statistical information is presented in Table 1. In the training phase of the scorer, we exclude samples corresponding to option 6 and reserve a portion of the data as the development set for hyperparameter tuning and model selection. Specifically, for the Restaurant datasets, we set aside 200 samples, and for the Laptop datasets, we allocate 300 samples. Consequently, this leaves around 1,000 training samples in the human-annotated datasets and approximately 2,000 training samples in the AI-annotated datasets.

heading: 4 Our Approach

heading: 4.1 Pseudo-label Scorer
text: The objective of the pseudo-label scorer is to score the match between a review and a pseudo-label. Previous efforts formalize this scoring task as a question-answering problem (Wang et al., 2021) or embed the discriminative matching token into the label (Mao et al., 2022). However, these methods struggle to effectively capture the interaction between reviews and pseudo-labels. Inspired by recent works in preference optimization (Rafailov et al., 2023; Yuan et al., 2023; Song et al., 2023), we utilize a generative model as the scorer. Given a review sentence x and a pseudo label y, their matching score is quantified by the conditional probabil-
ity assigned by the generative model:
s(x, y) ∝ p(y|x) = ∏
t
p(yt|y<t, x). (1)
Compared to previous methods, this approach integrates the likelihood of each token in the pseudolabel to derive its overall score, thereby providing a comprehensive and effective scoring.
Training. We optimize the pseudo-label scorer on the annotated comparison dataset with the rankingbased training objective. Specifically, we design a simple listwise objective5 as follows:
LLIST = − log p(yp|x)
Z , (2)
Z = p(yp|x) + ∑
yn
p(yn|x), (3)
where yp denotes the positive label, yn denotes the negative label, and Z is the normalization factor.
In addition to the comparison dataset, we also incorporate the original ASQP dataset to further enhance the training of the scorer. Labels from the original ASQP dataset are treated as additional positive labels and are combined with the positive labels from the comparison dataset. We additionally maximize the scores of these positive labels to enhance the scorer. The combined loss function is formulated as follows:
L =L1 + αL2, (4) L1 =E(x,Y)∼DCOMPLLIST(x,Y), (5) L2 =E(x,yp)∼DCOMP∪DASQP − log p(yp|x), (6)
where DCOMP represents the comparison dataset, DASQP represents the original ASQP dataset, Y denotes the set of several pseudo-labels of the sentence x, and α is a hyperparameter.

heading: 4.2 Self-Training with Data Filtering
text: Self-training (Scudder, 1965), a simple and classic semi-supervised technique, can be applied for data augmentation. It consists of three main steps: (1) training an initial model with the existing labeled dataset, (2) using this model to generate pseudolabels for unlabeled data, and (3) finally incorporating these pseudo-labeled data into the labeled dataset. However, this method inevitably introduces low-quality pseudo-labels, where the label does not accurately match the given review. To
5Besides the listwise objective, we also explore pointwise and pairwise objectives, which are presented in Appendix B.
overcome this issue, we implement a two-stage filtering process that leverages both the initial model and the pseudo-label scorer.
Confidence-based Filtering. We first use the confidence of the initial model in the pseudo-label as the measure of its quality. Thus, we filter out those samples with minimum confidence below a certain threshold. Formally, we retain samples (x, y) satisfying
[ min t p(yt|y<t, x) ] ≥ γ1, (7)
where γ1 is a hyper-parameter and is empirically set to 0.7.
Scorer-based Filtering. Next, we use the pseudolabel scorer to evaluate the remaining samples. We observe that pseudo-labels with low scores are consistently of poor quality. Besides, while samples with high scores generally exhibit good label quality, their sentences tend to be overly simple, offering limited helpfulness for subsequent model training. Therefore, we retain only those samples whose scores fall between thresholds γ2 and γ3, which can be formulated as follows:
γ2 ≤ s(x, y) ≤ γ3. (8)

heading: 4.3 Pseudo-label Scorer as Reranker
text: Reranking is originally a concept in information retrieval, referring to the process of rescoring and reranking preliminary candidate results. Ma et al. (2023) show that incorporating a reranking step can enhance performance in information extraction tasks. In this paper, we claim that our pseudo-label scorer can serve as such a reranker. Specifically, for a given review, we first utilize an ASQP model to generate four candidate labels via beam search and then select the best one from these candidates using our pseudo-label scorer. The selected candidate is utilized as the final output.

heading: 5 Experiments

heading: 5.1 Experiment Setup
text: Datasets. We evaluate our approach on four public ASQP datasets. These datasets originate from the SemEval Challenges (Pontiki et al., 2015, 2016) and Amazon platform during 2017 and 2018. The quad-level annotations are provided by Cai et al. (2021) and Zhang et al. (2021a). Detailed statistics of these datasets are presented in Table 2. Besides,
to train the pseudo-label scorer, we construct several comparison datasets, the statistics of which can be found in Table 1.
Implementation Details. We utilize T5-large (Raffel et al., 2020) as the backbone for our pseudolabel scorer. During the training phase, we set both the batch size and the number of training epochs to 10. For other hyperparameters, including the learning rate and α, we perform a simple hyperparameter search. Once the scorer is trained, we apply it to score and rank the pseudolabeled samples6. For datasets ACOS-Rest, ASQP-Rest15, and ASQP-Rest16, we retain samples with scores falling within the top 10% to 40%; for the ACOS-Laptop dataset, this range is set from 20% to 50%. From these retained samples, we randomly select 10,000 samples and merge them with the original labeled dataset to form the augmented dataset. To reduce the impact of randomness, we run our approach five times and report the average results.
Baselines. To validate the effectiveness of the proposed approach, we integrate it into two typical ASQP methods: GAS (Zhang et al., 2021b) and MUL (Hu et al., 2023b). We run these two methods on the augmented dataset and incorporate a reranking step during the inference phase to enhance the predictions. We also benchmark our approach against a range of other methods, including EXTRACT-CLASSIFY (Cai et al., 2021), PARAPHRASE (Zhang et al., 2021a), SEQ2PATH (Mao et al., 2022), DLO/ILO (Hu et al., 2022), LEGOABSA (Gao et al., 2022), MVP (Gou et al., 2023), GENDA (Wang et al., 2023), and CHATGPT (fewshot) (Xu et al., 2023).
6The source for the pseudo-labeled data is identical to that for the comparison data, originating from the Yelp Dataset and Amazon Laptop Dataset. But there is no overlap between them. The ASQP model used for pseudo-labeling is GAS (Zhang et al., 2021b).

heading: 5.2 Analysis of Pesudo-label Scorer
text: Given the importance of the pseudo-label scorer in our framework, we first undertake an analysis of it, focusing on two key aspects: its model architecture and the training dataset.
Model Architecture. We use the conditional likelihood the generative model assigns to a pseudolabel as its scoring metric. To examine the effectiveness of our approach, we conduct experiments on two human-annotated comparison datasets and benchmark our approach against previous methods (Wang et al., 2021; Mao et al., 2022). As Table 3 illustrates, previous methods, especially the question-answering method, perform poorly in the ASQP task. In contrast, our approach achieves a significant advantage, demonstrating its effectiveness.
Comparison Dataset. We conduct experiments to compare different annotation schemes and list the results in Table 4. We have the following observa-
tions. (1) Utilizing humans or AI to annotate the comparison data is crucial, as their performance is noticeably superior to that without annotation. Particularly, allowing human annotators to write a label when no option is suitable can notably enhance performance. (2) Training the scorer with a combination of the comparison data and the original ASQP dataset is more effective than using the comparison data alone. (3) AI-annotated comparison data can achieve even better results than human-annotated comparison data.
We conduct a further analysis of AI Annotation. Table 5 presents the consistency between AI- and human-annotated data. Although the consistency is not very high statistically, considering the subjective nature of this task, the quality of AI annotation is acceptable. Additionally, a significant advantage of AI annotation lies in its cost-effectiveness relative to human annotation, enabling the efficient acquisition of a large amount of annotated data.
Figure 2 illustrates the performance trends of human- and AI-annotated data relative to their quantities. Although AI-annotated data exhibits
lower performance at the same quantity, the scalability of AI annotation allows it to catch up and potentially exceed human-annotated data’s performance when using more data. For instance, more than 2,000 AI-annotated samples can equal or outperform 1,000 human-annotated samples. Consequently, we can conclude that for the ASQP task, it is feasible to replace humans with AI to annotate the comparison data.

heading: 5.3 Analysis of Self-Training
text: Main Results. We develop a self-training framework using the pseudo-label scorer, with the experimental results presented in Table 6. According to these results, our approach substantially and consistently improves the performance of existing ASQP methods (Zhang et al., 2021b; Hu et al., 2023b). Specifically, GAS achieves F1-score improvements of 2.94%, 4.32%, 5.17%, and 5.96% across the four datasets, averaging at 4.60%; MUL achieves F1score improvements of 3.72%, 3.39%, 2.98%, and 4.36% across these datasets, averaging at 3.61%. Upon integrating our approach, both GAS and MUL outperform previous methods. These results
demonstrate the effectiveness of our approach. Furthermore, we have the following observations. (1) The two-stage filtering process, namely CS-FILTER, greatly enhances the effectiveness of self-training. In most datasets, it results in over 2% improvement compared to self-training alone, highlighting the importance of data filtering in the self-training framework. (2) Incorporating a rerank step can further improve performance, by around 1%. (3) Using AI-annotated data in downstream self-training can attain results comparable to those using human-annotated data. This further indicates the feasibility of replacing human annotators with AI for comparison data annotation. (4) ChatGPT performs poorly on the ASQP task, suggesting that using it directly for this task does not fully leverage its capabilities. Conversely, using it for comparison data annotation effectively exploits its strengths. (5) It can be noted that our filtering strategy offers relatively limited improvements on ACOS-Laptop. We attribute this to potential inconsistency between its ASQP annotations and our comparison annotations. A more detailed discussion is available in Further Analysis.
Effect of Match Score. Our approach relies on the match scores output by the pseudo-label scorer for data filtering. We conduct experiments to examine the impact of these scores on self-training performance. Figure 3 illustrates that the performance incrementally increases with increasing match scores. Nevertheless, beyond a certain threshold, further increases in match scores lead to a decline in performance. This phenomenon confirms our hypothesis that samples with too low scores suffer from poor label quality, adversely affecting model learning, and samples with too high scores tend to be overly simple, providing limited helpfulness for subsequent model training.
Effect of Data Quantity. The quantity of pseudolabeled samples is another important factor in the effectiveness of self-training. We conduct experiments to analyze its impact. As illustrated in Figure 4, there is an overall upward trend in performance with increased data quantity. Notably, this trend is more stable and pronounced following twostage filtering, underscoring the necessity of data filtering. Furthermore, we notice a decrease in self-training performance when the number of augmented samples exceeds 20,000. This suggests that there is a limit to improving performance by simply increasing data quantity. Balancing diversity and label quality to enhance the effectiveness of selftraining warrants further exploration in subsequent research.

heading: 5.4 Further Analysis
text: Comparison Data as Additional Labeled Data. One feasible approach for utilizing comparison data is to treat each sample along with its positive label as an additional labeled ASQP sample. We analyze the effectiveness of this approach, and
the results in Table 7 reveal: (1) this approach can improve performance, with human-annotated comparison data outperforming AI-annotated data; (2) under conditions of equal data volume, it is superior to self-training without data filtering, indicating that the quality of comparison data is better than that of pseudo-labeled data; and (3) however, it falls significantly short of self-training with data filtering. These findings suggest that utilizing comparison data to train a pseudo-label scorer is more effective than simply treating it as additional labeled data.
Pseudo-label Scorer as the ASQP Model. The pseudo-label scorer is architecturally a generative model and can potentially be used as an ASQP
model. We evaluate this possibility and list the results in Table 8. A surprising finding is that directly using the scorer to predict quads achieves good performance, though it generally falls short of using it for filtering and ranking. This suggests that, besides training the scorer, leveraging comparison data to enhance the ASQP model is a promising direction, deserving of in-depth investigation in future research.
Assessing Label Quality in ASQP Data. Beyond assessing pseudo-labeled data, our scorer can assess the quality of existing ASQP data. We conduct a statistical analysis of match scores for ASQP samples and list the results in Table 9. Our analysis reveals relatively low match scores for the ACOS-laptop dataset, suggesting either poor annotation quality or low consistency with our comparison data. We manually review 100 samples with match scores low 0.1 and find that 73% of the data contradicts the annotation guidelines, including 44% with errors in aspect category annotations, 8% in aspect or opinion term annotations, and 6% in sentiment annotations. Moreover, we experiment with removing samples with low match scores. The results presented in Table 10 show that this removal not only preserves model performance but enhances it. These findings indicate that our scorer is an effective tool for assessing label quality in existing datasets and that the removal of
low-quality samples is advantageous.
Analysis of Reranking. We present the analysis of the reranking step in Appendix C.

heading: 6 Conclusions
text: In this paper, we introduce a pseudo-label scorer for the Aspect Sentiment Quad Prediction (ASQP) task to reduce mismatches in data augmentation. We propose that the effectiveness and reliability of this scorer hinge on two critical aspects: the quality of the training dataset and its model architecture. To this end, we create both human- and AI-annotated comparison datasets and propose a scoring method based on a generative model. Upon developing this scorer, we apply it to data filtering in a self-training framework and further employ it as a reranker to enhance ASQP models. Detailed experiments and analysis demonstrate the effectiveness of our comparison datasets and the proposed architecture. Furthermore, experimental results on four public ASQP datasets reveal that our scorer significantly and consistently improves the performance of existing methods.

heading: Acknowledgements
text: We thank the anonymous reviewers for their valuable suggestions to improve the quality of this work. We also thank Qianlong Wang for the enlightening discussions that significantly contributed to this work. This work was partially supported by the National Natural Science Foundation of China 62176076, Natural Science Foundation of Guangdong 2023A1515012922, the Shenzhen Foundational Research Funding JCYJ20220818102415032, the Major Key Project of PCL2021A06, and Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies 2022B1212010005.
Limitations
While our approach significantly enhances the effectiveness of data augmentation and improves the performance of existing ASQP models, it also suffers from the following limitations:
• Data augmentation generally comprises two pivotal components: data synthesis and quality control. While this paper focuses primarily on the latter, the former is equally vital for the success of data augmentation. Given that models trained on limited labeled data may underperform in certain categories or contexts, targeted data synthesis can mitigate these issues. A comprehensive exploration of both data synthesis and quality control is essential for developing an effective and robust data augmentation framework.
• The implementation of our approach necessitates manually annotated comparison data. Although we could use large language models to replace human annotators, crafting and refining prompts still demands meticulous human expertise and is notably time-intensive.
We argue that these limitations offer promising directions for future research.

heading: A Details of AI Annotation
text: We employ ChatGPT7 to replace humans for comparison data annotation. The prompt is depicted in Figure 5. We continuously refine the guidelines and demonstrations in the prompt based on the annotation results. Furthermore, to further enhance annotation quality, we adopt three strategies:
• Self-consistency: For each sample, we input it into ChatGPT twice. We keep the sample if the results are consistent.
• Self-assessment: We require ChatGPT to evaluate its confidence level on a scale from 1 to 5 after each judgment. We retain only those samples with a confidence level is 5.
7Available at https://chat.openai.com/. The specific model used is gpt-4-1106-preview.

heading: 1. First, check if the review is relevant and expresses any sentiment. {other guidelines}

heading: 1. Review: "There was only 1 time that food came in under a half hour." Pseudo-label options:
text: • Rationale augmentation: We instruct ChatGPT to provide reasoning and explanation before making its judgment.
Additionally, to reduce annotation costs, we integrate four samples into one prompt and annotate them at once.

heading: B Ranking Objectives for Training Scorer
text: We optimize the pseudo-label scorer on the annotated comparison dataset and explore three rankingbased training objectives: pointwise, pairwise, and listwise approaches. The pointwise approach classifies positive and negative samples separately and
can be formulated as follows:
LPOINT =− log p(yp|x) − ∑
yn
log(1− p(yn|x)), (9)
where yp denotes the positive label, and yn denotes the negative label. The pairwise approach focuses on the relative quality of labels. We implement two pairwise training objectives, detailed as follows:
LPAIR1 =− ∑
yn
log σ [ β log
p(yp|x) p(yn|x)
] , (10)
LPAIR2 = ∑
yn
max(0, p(yn|x)− p(yp|x)), (11)
where β is a hyper-parameter. Lastly, the listwise approach optimizes the ranking of the entire list. We design a simple listwise training objective, as outlined in Equation 2.
B.1 Experiment Results
Experimental results in Table 11 reveal that pointwise and listwise objectives outperform two pairwise objectives, with the listwise objective being slightly better overall. Consequently, we adopt the listwise objective as the default training objective.

heading: C Analysis of Reranking
text: We apply our pseudo-label scorer as a reranker to rescore the candidate labels generated by the ASQP model, selecting the highest-scoring one as the final result. Table 12 shows that this reranking step
significantly improves performance on the ASQP task, resulting in an average F1 improvement of 1.41%. Additionally, if we consider a hypothetical perfect reranker that always selects the optimal candidate, Table 12 shows that the performance gain could reach up to 15.75%. This significant potential boost underscores the value of further exploring the reranking step.
Furthermore, we rank the four candidate labels obtained via beam search according to their confidence and then analyze the distribution of the best labels and those preferred by our scorer. As illustrated in Table 13, in fewer than 70% of cases, the candidate label with the highest confidence is considered the best. In comparison, our scorer tends to favor candidates with higher confidence, highlighting areas for further improvement in this reranking step.

heading: Future Work
text: We argue that these limitations offer promising directions for future research