It is Simple Sometimes: A Study On Improving Aspect-Based Sentiment Analysis Performance (2024)

Abstract: Aspect-Based Sentiment Analysis (ABSA) involves extracting opinions from textual data about specific entities and their corresponding aspects through various complementary subtasks. Several prior research has focused on developing ad hoc designs of varying complexities for these subtasks. In this paper, we build upon the instruction tuned model proposed by Scaria et al. (2023), who present an instruction-based model with task descriptions followed by in-context examples on ABSA subtasks. We propose PFInstruct, an extension to this instruction learning paradigm by appending an NLP-related task prefix to the task description. This simple approach leads to improved performance across all tested SemEval subtasks, surpassing previous state-of-the-art (SOTA) on the ATE subtask (Rest14) by +3.28 F1-score, and on the AOOE subtask by an average of +5.43 F1-score across SemEval datasets. Furthermore, we explore the impact of the prefix-enhanced prompt quality on the ABSA subtasks and find that even a noisy prefix enhances model performance compared to the baseline. Our method also achieves competitive results on a biomedical domain dataset (ERSA).

## introduction user-generated reviews on e-commerce and social media platforms benefit both consumers and stakeholders. with the exponential growth of data, developing reliable tools for understanding the sentiment of online review texts is essential to moderate online content, enable effective decision-making and customer satisfaction. @xcite proposed aspect-based sentiment analysis (absa) as a step towards fine-grained sentiment analysis of specific aspects. absa involves the detection of opinions subtask definition prefix examples sample input + + + model input re task ner task noise the service was slow, but the pasta was excellent. s 1 negative s 2 positive s: o 2 o 1 a 1 a 2 ac: food semeval ersa sentihood s: sentence a i : aspect o i : opinion s i : sentiment ac: aspect category def. atsc def. ersa (s, a 2 , ac) s 1 (s, a 1 , a 2 ) s 1 def. ate def. atsc def. aooe subtask s (s, a 1 ), (s, a 2 ) (s, a 1 ), (s, a 2 ) (a 1 , a 2 ) s 1 , s 2 o 1 , o 2 input output figure 1 : illustration of model input and absa subtasks examined in this paper. the prefix can vary between nlp-related tasks (instruction) or textual noise (random words), followed by the subtask definition, few examples and the corresponding sample input for each subtask. the model is expected to follow the instructions and generate a prediction. subtasks belong to three distinct data sources: semeval, ersa and sentihood from different domains. in this paper, we propose pfinstruct, an exten-sion to the instructabsa framework with the introduction of prefix prompt. specifically, we append a prefix to the task definition, extend the evaluation to domains like biomedicine and urban neighbourhoods, and formulate all the subtasks as a generative task. the prefix aims at instructing the model on a related nlp task, namely relation extraction (re) or named entity recognition (ner), so the target text s is seen on a different task context. we postulate this approach helps to collect richer semantic information about the main entities in s, which allows the model to make a more informed prediction about the absa subtasks. we also consider to use a randomly generated (noise) prefix. we observe that not only it boosts -to a lesser degree-average performance, but it also makes the model more robust to out-of-domain data. contributions we introduce a simple approach to solve absa subtasks, that is, a prefix prompt followed by instructions. our approach outperforms previous sota on several tasks despite being based on a 200m model. we conduct extensive analysis on five subtasks from different domains: customer reviews from semeval 2014, 15 and 16 @xcite @xcite , on a biomedical domain dataset, ersa (young and akujuobi, 2023), and on user comments from sentihood @xcite . we assess the effect of the prefix prompt in terms of prompt quality (re, ner or noise) and domain generalization (out-of-domain performance). we make our code publicly available to ensure reproducibility and foster future research. foot_0 ## related work current state-of-the-art (sota) solutions on se-meval datasets include lsa @xcite on atsc, which leverage the use of gradient descent to design a differential-weighted approach, bartabsa @xcite on aooe, based on an end-to-end generative framework, and @xcite on sentihood (atsc). on a similar line of research to @xcite and our work, @xcite propose it-mtl, multitask prompting with instructional prompts to tackle absa subtasks as a question-answering problem. for a review of popular absa datasets, we refer to @xcite . to compare to related work @xcite @xcite @xcite 3 background and methodology 3.1 background: absa absa subtasks can be classified into single output and compound output subtasks. in single output subtasks such as ate (aspect-term extraction), atsc (aspect-term sentiment classification), and aooe (aspect-opinion extraction), the output is limited to either the aspect a, opinion o or sentiment s from a given text s. compound output subtasks ask for a combination of the {a, o, s} entity types. many researchers @xcite @xcite focus only on the former. for convenience, we do the same. 2 in addition, we evaluate our method on ersa (young and akujuobi, 2023) and sentihood (atsc) @xcite tasks. ersa is an extension of the absa task, where, given a text s and two aspects (or entities) a 1 and a 2 , where a 1 ̸ = a 2 , the goal is to determine the sentiment polarity s of the relationship between the aspects given s. ersa targets biomedical texts but it does not require to have factual knowledge about the entities. sentihood, defined as an extension of atsc, requires to classify the sentiment s towards each aspect a i of one or more aspect categories ac. appendix a describes the datasets in more detail. ## results we present the main results of our experimental set-up. report macro-averaged f1-score averaged across five random initialization seeds and the standard deviation. details about fine-tuning settings are provided in appendix a. ## conclusion in this paper, we present pfinstruct, a simple yet effective prefix prompting strategy to instruction fine-tune a language model on absa subtasks. we analyse the impact of the prefix prompt's quality on in-domain and out-of-domain data and observe that even a random prefix improves average model performance compared to the instructabsa baseline. we evaluate our method on domains such as customer reviews, biomedical text and user comments, and show that it outperforms previous sota approaches on most of the tasks tested and achieves competitive performance (f1-score) in the rest.