ZZU - NLP at SIGHAN -2024 dim ABSA Task: Aspect-Based Sentiment Analysis with Coarse-to-Fine In-context Learning (2024)

Abstract: The DimABSA task requires fine-grained sentiment intensity prediction for restaurant reviews, including scores for Valence and Arousal dimensions for each Aspect Term. In this study, we propose a Coarse-to-Fine In-context Learning (CFICL) method based on the Baichuan2-7B model for the DimABSA task in the SIGHAN 2024 workshop. Our method improves prediction accuracy through a two-stage optimization process. In the first stage, we use fixed in-context examples and prompt templates to enhance the model’s sentiment recognition capability and provide initial predictions for the test data. In the second stage, we encode the Opinion field using BERT and select the most similar training data as new in-context examples based on similarity. These examples include the Opinion field and its scores, as well as related opinion words and their average scores. By filtering for sentiment polarity, we ensure that the examples are consistent with the test data. Our method significantly improves prediction accuracy and consistency by effectively utilizing training data and optimizing in-context examples, as validated by experimental results.

## introduction aspect-based sentiment analysis (absa) @xcite @xcite is a critical nlp research topic that aims to identify the aspects of a given entity and analyze the sentiment polarity associated with each aspect. absa involves predicting tuples of sentiment elements for a given text, with four main elements constituting the focus of absa research: aspect term (a), aspect category (c), opinion term (o), and sentiment polarity (s) @xcite . early studies on absa primarily focused on single sentiment elements such as aspect term @xcite , aspect category (zhou * corresponding author @xcite , or sentiment polarity @xcite . however, recent research has introduced compound absa tasks involving multiple associated elements. these include aspect sentiment triplet extraction (aste) @xcite @xcite @xcite @xcite , which extracts three elements in a triplet-aspect/target term, opinion term, and sentiment polarity. furthermore, aspect sentiment quadruple prediction (asqp) @xcite @xcite @xcite extends aste by including an additional aspect category, thus constructing a quadruple. in contrast to representing affective states as discrete classes (i.e., polarity), there is also a dimensional approach that represents affective states as continuous numerical values, such as in the valence-arousal (va) space @xcite , providing more fine-grained emotional information @xcite . for example, in the sentence "独家的鲔鱼抹酱超好吃。", the corresponding elements are "鲔鱼抹酱" (aspect term), "食物#品 质" (aspect category), "超好吃" (opinion term), and "7.5#7.25" (valence#arousa score). resently, large language models (llms) @xcite have shown an impressive few-shot ability on several nlp tasks. to expect llms to perform better on few-shot tasks, in-context learning (icl) @xcite paradigm is becoming a flourishing research direction. this paradigm can generate a prediction of the test input by conditioning on few-shot input-output examples (also known as in-context examples or demonstrations), without requiring any updates to parameters. previous studies @xcite found that llms are highly sensitive to the choice of in-context examples. one typical strategy for retrieving helpful in-context examples is to leverage the overall semantic similar-ity between the candidate examples and test input. further research has shown that retrieving highly relevant examples across multiple dimensions has achieved significant performance improvements in multi-domain absa tasks @xcite . supervised fine-tuning (sft) is a method that involves further training a pre-trained model using a labeled dataset to achieve better performance on specific tasks. in-context learning helps the model understand the task by providing a few examples during inference, but its performance is often limited by the selection and number of examples. sft, on the other hand, directly trains on a large amount of labeled data, allowing the model to deeply understand various aspects of the task, thereby exhibiting higher accuracy and consistency in practical applications and achieving good performance on specific tasks @xcite . our study addresses the dimabsa task at the sighan 2024 workshop by proposing a two-stage context learning method based on the baichuan2-7b @xcite model to improve the accuracy of fine-grained sentiment intensity prediction for restaurant reviews. our work consists of two main stages: in the first stage, we use fixed context examples to train the model, enhancing its ability to recognize sentiment elements. in the second stage, we utilize the chinese bert @xcite to encode the opinion field and select the most similar training data as new context examples based on similarity calculations, thereby further improving the model's prediction accuracy and granularity. experimental results show that our method significantly enhances the model's performance in both valence and arousal dimensions and effectively reduces sentiment polarity bias. overall, our approach provides an efficient solution for the dimabsa task and offers valuable insights for the optimization of future fine-grained sentiment analysis models. ## background the chinese dimensional aspect-based sentiment analysis (dimabsa) @xcite shared task is part of the sighan 2024 workshop foot_0 . this task focuses on providing fine-grained sentiment intensity predictions for each extracted aspect of a restaurant review. the four sentiment elements are defined as follows: aspect term (a): denotes an entity indicating the opinion target. if the aspect is omitted and not mentioned clearly, "null" is used to represent the term. aspect category (c): represents a predefined category for the explicit aspect within the restaurant domain. the categories are based on the semeval-2016 restaurant dataset @xcite and include twelve categories, each split into an entity and attribute using the symbol "#". opinion term (o): describes the sentiment words or phrases related to the aspects. sentiment intensity (i): reflects the sentiments using continuous real-valued scores in the valence-arousal dimensions. valence indicates the degree of pleasantness (positive or negative feelings), while arousal indicates the degree of excitement or calmness. both dimensions use a nine-degree scale, where 1 denotes extremely high-negative or low-arousal sentiment, 9 denotes extremely high-positive or high-arousal sentiment, and 5 denotes neutral or medium-arousal sentiment. this task aims to evaluate the capability of automatic systems for chinese dimensional absa and is divided into three subtasks: subtask 1: intensity prediction: focuses on predicting sentiment intensities in the valence-arousal dimensions. given a sentence and a specific aspect, the system should predict the valence-arousal ratings. subtask 2: triplet extraction: aims to extract sentiment triplets composed of three elements (aspect, opinion, intensity) from a given sentence. subtask 3: quadruple extraction: aims to extract sentiment quadruples composed of four elements (aspect, category, opinion, intensity) from a given sentence. our team chose to participate in the more challenging second and third subtasks, and we achieved third place in the evaluation task. ## system overview we use baichuan2-7b as the base model and propose a two-stage context learning method to improve prediction accuracy in the dimabsa task. this method incrementally optimizes the model output through preliminary and refined prediction stages, fully utilizing the information in the training data. our framework is shown in figure 1 . baichuan2 @xcite is a chinese and english bilingual language model. it achieved the best performance among models of the same size on standard benchmarks(c-eval @xcite , mmlu @xcite ). these examples include the opinion field and its scores, as well as related opinion words and average scores. sentiment polarity filtering ensures that the in-context examples are consistent with the test data. finally, these new in-context informations are input into the model along with the test data for re-prediction, yielding optimized quadruple results. ## conclusion this study proposes a two-stage context learning method based on the baichuan2-7b model for the dimabsa task at the sighan 2024 workshop. the task requires fine-grained sentiment intensity prediction for restaurant reviews. in the first stage, we enhance the model's sentiment element recognition ability using fixed context examples. in the second stage, we utilize bert to encode the opinion field and select the most similar training data based on similarity calculation as new context examples. these relevant examples improve the accuracy of sentiment intensity prediction. experimental results show that our two-stage method significantly enhances the accuracy and granularity of predictions. the method effectively utilizes training data and reduces sentiment polarity bias. overall, our approach provides an efficient solution for the dimabsa task and offers valuable insights for optimizing future models for fine-grained sentiment analysis.