Is Compound Aspect-Based Sentiment Analysis Addressed by LLM s? (2024)

Abstract: Aspect-based sentiment analysis (ABSA) aims to predict aspect-based elements from the given text, mainly including four elements, i.e., aspect category, sentiment polarity, aspect term, and opinion term. Extracting pair, triple, or quad of elements is defined as compound ABSA. Due to its challenges and practical applications, such a compound scenario has become an emerging topic. Recently, large language models (LLMs), e.g. ChatGPT and LLaMA, present impressive abilities in tackling various human instructions. In this work, we are particularly curious whether LLMs still possess superior performance in handling compound ABSA tasks. To assess the performance of LLMs, we design a novel framework, called ChatABSA. Concretely, we design two strategies: constrained prompts, to automatically organize the returned predictions; post-processing, to better evaluate the capability of LLMs in recognition of implicit information. The overall evaluation involves 5 compound ABSA tasks and 8 publicly available datasets. We compare LLMs with few-shot supervised baselines and fully supervised baselines, including corresponding state-of-the-art (SOTA) models on each task. Experimental results show that ChatABSA exhibits excellent aspect-based sentiment analysis capabilities and overwhelmingly beats few-shot supervised methods under the same few-shot settings. Surprisingly, it can even outperform fully supervised methods in some cases. However, in most cases, it underperforms fully supervised methods, and there is still a huge gap between its performance and the SOTA method. Moreover, we also conduct more analyses to gain a deeper understanding of its sentiment analysis capabilities.

## introduction sentiment analysis (sa) is the umbrella term for techniques and approaches that analyze peo-* corresponding author. ## experimental results we conduct experiments using both chatgpt and llama-3. since the experimental results and conclusions of both models are highly consistent, we primarily focus on analyzing the results of chat-gpt, with chatabsa defaulting to chatgpt. ## conclusion in this work, we explore the boundaries of llms' capabilities in compound absa by comparing fully supervised methods and few-shot supervised methods. because llms' predictions are often unstable, we have designed a more rational framework called chatabsa. this framework aims to better evaluate llms' performance on compound absa. chatabsa exhibits excellent aspect-based sentiment analysis capabilities and overwhelmingly beats few-shot supervised methods under the same few-shot settings. surprisingly, it can even outperform fully supervised methods in some cases. however, in most cases, it underperforms fully supervised methods, and there is still a huge gap between its performance and the sota method. furthermore, it is still challenging and struggling for llms to predict implicit elements. in summary, although llms possess strong language comprehension and can accurately follow instructions for specific tasks, it does not perform well on compound absa tasks. we hope that our research will inspire future research in llms and aspect-based sentiment analysis.