I nstruct ABSA : Instruction Learning for Aspect Based Sentiment Analysis (2024)

Abstract: We introduce InstructABSA, an instruction learning paradigm for Aspect-Based Sentiment Analysis (ABSA) subtasks.Our method introduces positive, negative, and neutral examples to each training sample, and instruction tune the model (T k -Instruct) for ABSA subtasks, yielding significant performance improvements. Experimental results on the Sem Eval 2014, 15, and 16 datasets demonstrate that InstructABSA outperforms the previous state-of-the-art (SOTA) approaches on Term Extraction (ATE), Sentiment Classification(ATSC) and Sentiment Pair Extraction (ASPE) subtasks.In particular, InstructABSA outperforms the previous state-of-the-art (SOTA) on the Rest14 ATE subtask by 5.69% points, the Rest15 ATSC subtask by 9.59% points, and the Lapt14 AOPE subtask by 3.37% points, surpassing 7x larger models.We get competitive results on AOOE, AOPE, AOSTE, and ACOSQE subtasks indicating strong generalization ability to all subtasks. Exploring sample efficiency reveals that just 50% train data is required to get competitive results with other instruction tuning approaches. Lastly, we assess the quality of instructions and observe that InstructABSA’s performance experiences a decline of ~10% when adding misleading examples

## introduction aspect based sentiment analysis (absa) plays a vital role in understanding the fine-grained sentiments expressed by users @xcite . as illustrated in figure 1 , absa extracts aspects and classifies the aspect's sentiment polarity by extracting and understanding the author's opinions. instruction learning paradigm @xcite 1 experiments and results are available at https:// github.com/kevinscaria/instructabsa † currently in amazon (the work was done prior to joining amazon) ♢ currently in google deepmind figure 1 : illustration of the six absa subtasks where s i is the i th sentence, a i are the aspect terms, sp i are the sentiment polarities and o i is the opinion terms. @xcite has significantly improved the reasoning abilities of large language models (llms) and has shown impressive results across various tasks @xcite . owing to its previous success, we propose instructabsa, instruction learning for aspect based sentiment analysis (absa). our approach involves further instruction tuning of the tk-instruct model @xcite to address six subtasks of absa as shown in fig. 1 . we add instruction prompts specific to the downstream absa subtasks in the form of task definitions, followed by positive, negative, and neutral examples. we carried out extensive experiments on the se-meval 2014, 15, and 16 datasets @xcite @xcite , and the dataset by @xcite for the aoste subask, which comprises the laptops and restaurants domain. across the subtasks in both domains, instructabsa outperforms sota approaches. specifically, for the 2014 ate subtask, we obtain f1-score of 92.3 and 92.76 (lapt14, rest14), surpassing sota by 4.37% and 5.69% points respectively. for the atsc subtask, instructabsa attains an accuracy of 84.50 in the rest15 dataset exceeding the previous results by 9.59% points. in the rest14 dataset atsc subtask, our approach gets a competitive accuracy score of 86.25 compared to the sota of 90.86. for the aspe subtask, instructabsa achieves f1-score of 79.34 and 79.47 (lapt14, rest14), outperforming sota by 3.37% and 1.4% points, respectively. we get competitive results on aooe and aoste approaches as well ( §4). we conduct a thorough analysis along several lines of enquiry. we showcase sample efficiency of instructabsa by achieving competitive scores using roughly 20% of training samples as compared to @xcite 's instruction tuning approach. we compare instructabsa with finetuning methods such as low-rank adaptation (lora) @xcite to find that there is a sizebale gap of ∼ 20%. to understand the effect of different instructions for absa, we change the prompts on the lines of definition and task manipulation. we find that delusive examples roughly decrease the approaches results by ∼ 10% giving a strong evidence of the impact of instructions on instructabsa. we also provide evidence of crossdomain and joint-domain generalizations arising as part of our proposed approach. contributions:(a) we introduce instructabsa, which achieves performance gains on absa subtasks of semeval 2014,15 and 16 datasets, surpassing the previous sota models. (b) despite using a 200m model, instructabsa outperforms or get competitive results over the prior sota models with 1.5b parameters. (c) finally, we provide an analysis of the impact of our method in terms of sample efficiency, adapter methods, effect of instruction and domain generalization. ## instructabsa: instruction learning for absa we describe the mathematical formulation of absa subtasks and the proposed approach. let s i represent the i th review sentence in the training sample, where:s @xmath0, w foot_0 i , ..., w n i with n as the number of tokens in the sentence. each s i contains a set of aspect terms denoted by in these equations, lm represents the language model, and the corresponding inputs and outputs are defined accordingly. as part of our approach, we instruction tune lm subtask by prepending taskspecific instruction prompts inst to each input sample to arrive at lm inst subtask . here, @xmath1, we use "'tk-instruct-base"' as the model. the def inition involves the task definition for each subtask. contrary to the standard instruction tuning prompts proposed by @xcite , p ositiveexample and n egativeexample here represent examples that have a positive and negative sentiment example respectively. additionally, we introduce n eutralexample which is an example that has neutral sentiment respectively( §f). ## experimental setup we use the tk-instruct-base-def-pos 2 as the instruction-tuned model lm inst . we use two configurations of instructions as prompts for our experiments. instructabsa-1 has the instruction prompt that includes the definition of the absa subtasks followed by 2 positive examples for the respective task. instructabsa-2 has the definition followed by 2 positive, negative, and neutral examples. dataset: semeval 2014,15 and 16 datasets are used for our experimentation. the dataset is used as a benchmark for absa tasks and has customer reviews from three domains; laptops (lapt14), hotels (hotel15), and restaurants (rest14, rest15, and rest16). more details can be found in §c. evaluation metric: following previous approaches @xcite , we use the micro f1-score for ate, aope, aooe, aope, aoste, and the accuracy for atsc. ## conclusion we proposed instructabsa, an instruction-tuned modeling approach for all subtasks of absa. our findings show that instructabsa surpassed the previous scores on several tasks and achieved competitive scores on the rest using a significantly smaller model than previous approaches. we further analyzed the performance of the approach along several lines of enquiry revealing several interesting findings.