A Weak Supervision Approach for Few-Shot Aspect Based Sentiment Analysis (2024)

Abstract: We explore how weak supervision on abundant unlabeled data can be leveraged to improve few-shot performance in aspect-based sentiment analysis (ABSA) tasks. We propose a pipeline approach to construct a noisy ABSA dataset, and we use it to adapt a pre-trained sequence-to-sequence model to the ABSA tasks. We test the resulting model on three widely used ABSA datasets, before and after fine-tuning. Our proposed method preserves the full fine-tuning performance while showing significant improvements (15.84 absolute F1) in the few-shot learning scenario for the harder tasks. In zero-shot (i.e., without fine-tuning), our method outperforms the previous state of the art on the aspect extraction sentiment classification (AESC) task and is, additionally, capable of performing the harder aspect sentiment triplet extraction (ASTE) task.

## introduction aspect based sentiment analysis (absa) is a finegrained variant of sentiment analysis @xcite @xcite @xcite @xcite , where the task is to predict the sentiment expressed towards an entity or a certain aspect of an entity, instead of just the sentence-level sentiment (e.g., traditional sentiment analysis tasks @xcite ; dos santos and de c. gatti, 2014)). for illustration, for a review the pizza was great, but the service was terrible, a sentence-level sentiment analysis model might identify the sentiment as neutral. the need for absa stems from such complex interactions between the target and the polarity of the sentiment @xcite ). an absa model has to identify the sentiment towards pizza as positive, and service as negative, for a holistic understanding of the text. furthermore, * work done during internship at aws ai labs † work done while at aws ai labs absa tasks can include the identification of the opinion terms (i.e. great, terrible), and the aspect categories (i.e. food, service) @xcite . although traditionally considered as a structured prediction task in the absa literature, recent works have shown how sequence-to-sequence (seqto-seq) models can be effective in these tasks with a generative approach @xcite . such approaches leverage the knowledge gained from one task to seamlessly perform well in another. as such, we build upon the instruction tuning with multi-task learning approach @xcite and address the following five absa tasks: (i) aspect-term extraction (ae), (ii) aspect-term extraction and sentiment classification (aesc), (iii) target aspect sentiment detection (tasd), (iv) aspect sentiment triplet extraction (aste), and (v) aspect sentiment quadruple prediction (asqp). sentence-level sentiment annotations are comparatively cheaper and are available at scale through automated proxies (e.g., ☀ or ☀☀ become negative, ☀☀☀☀ or ☀☀☀☀☀ become positive, in the review corpora @xcite ). on the contrary, absa requires understanding at sub-sentence level with multiple words or phrases being related to each other, making it prohibitively costly to annotate at scale. 1 however, the abundance of generic review data presents a promising opportunity to improve the performance of a pre-trained language model (plm) beyond simply fine-tuning it on the small annotated absa corpora. towards this end, we first construct a noisily annotated absa corpus out of generic customer review data without any direct supervision. we utilize this noisy corpus to pre-train a seq-to-seq model on multiple absa tasks. we show that such models are capable of learning in zero/few-shot in final downstream absa tasks. our contributions are the following: (i) we propose a weakly supervised method to obtain annotations for three out of the five absa tasks explored in the literature; @xcite we introduce a pre-training step to improve the few-shot performance on the downstream task of plms; (iii) we comprehensively evaluate our proposed method in three scenarios (full fine-tuning, few-shot, and zero-shot learning), yielding as much as 15.84% f1 improvement over the sota baselines. we release all the sources to reproduce the datasets and results presented foot_2 . ## related work aspect-based sentiment analysis has received tremendous attention in the past years @xcite @xcite , either handling single tasks, such as aspect term extraction @xcite @xcite , aspect category detection @xcite , aspect sentiment classification @xcite @xcite , or handling compound tasks @xcite @xcite . for the latter group, it typically includes either a pipeline approach @xcite or an end-to-end (e2e) approach @xcite . in the pipeline approach the final prediction is constructed using the output of multiple components. the disadvantage of such models is that the error is propagated throughout the system @xcite . in the e2e approach, the model learns the interactions jointly between the multiple prediction tasks, which is believed to improve the final performance @xcite . our proposed approach falls in this category. typical e2e approaches include: (i) treating it as a token classification task @xcite @xcite , (ii) framing it as a machine reading comprehension task @xcite , natural language inference task @xcite , or as a language generation task @xcite @xcite . our proposed approach treats the absa tasks as a generation task, similar to @xcite . we build upon the paradigm called instruction tuning with in multi-task learning (it-mtl), introduced in @xcite , resulting in a single model capable of handling different absa tasks. however, none of these methods takes advantage of the vast amount of review data available, other than just pre-training on them with some generic language modeling objectives. despite impressive generalization capabilities shown by llm based systems e.g., chatgpt, gpt-4 they reportedly struggle to perform well on these tasks @xcite . ## method we introduce an additional step in the classical @xmath0, transforming it into @xmath1. we propose an approach for building a weakly annotated dataset for the intermediate napt step. we use this noisy dataset to enhance the knowledge of a pretrained model with the intuition that exposing the model to tasks which are well aligned with the final downstream task, improves the performance. we then consider this as the backbone base model, and finetune it on the downstream task as usual. our proposed approach is applicable to any generic seq-to-seq model. ## experiments we compare against state-of-the-art methods on three widely used absa datasets. we evaluate in three scenarios: (i) k-shot learning: where the model has access to at least k examples of each class, (ii) zero-shot evaluation: where the model has not seen any example at all from the goldannotated absa data, and (iii) full-training: where the model has access to the complete gold-standard training data. ## discussion in this section, we discuss a few important aspects of our approach apart from the main experiments. ## conclusion in this paper, we proposed to add an intermediate step in the @xmath2, called noisy absa pre-training. we motivate this newly introduced step with the hypothesis that exposing the model to tasks more aligned with the down-stream task will improve its performance, especially in low-data regimes such as in few-shot or complete zero-shot. we constructed a noisy dataset with a heuristic based pipeline approach consisting of four steps that utilize well-studied nlp resources and models. this resulting dataset serves as the training dataset for the noisy pretraining phase. we then evaluated with customer reviews from three datasets covering two domains, laptop (lap14) and restaurant (rest15, rest16), and obtained large improvements in the zero/fewshot cases while achieving similar performance under finetuning on full dataset. we also discussed caveats around introducing catastrophic forgetting of general purpose pre-trained language models through such noisy pre-training, and introduced a few regularization techniques to help alleviate it.