An Instruction Tuning-Based Contrastive Learning Framework for Aspect Sentiment Quad Prediction with Implicit Aspects and Opinions (2024)

Abstract: Aspect sentiment quad prediction (ASQP) is crucial in aspect-based sentiment analysis (ABSA). It involves identifying a text’s aspect,sentiment, opinion, and category. Existing methods have insufficiently explored how to effectively leverage the knowledge of pre-trainedlanguage models (PLMs) to handle implicit aspects and opinions, particularly in combinations such as implicit aspect & explicit opinion, explicit aspect & implicit opinion, and implicit aspect & implicit opinion. We introduce ITSCL, a framework leveraging Instruction Tuning and Supervised Contrastive Learning to improve aspect sentiment quad predictions, especially for implicit aspects and opinions. Implementing this approach presents several challenges. First, designing effective instructions and prompts to optimize the model’s training is difficult. Second, creating sentiment combination vectors with contrastive learning to enhance the model’s discrimination requires further investigation. To address these challenges, ITSCL combines instruction tuning with aligned PLM templates, enabling better knowledge acquisition and identification of implicit sentiments. Additionally, the contrastive learning framework enhances performance by using four fully connected layers to combine sentiments, aspects, opinions, and combinations, maximizing similarity for same-label representationsand minimizing it for different labels. Experimental results show our method significantly outperforms previous methods on benchmark datasets.

## introduction asqp aims to extract four key components from a given text: aspect, sentiment, opinion, and category (as shown in figure 1 ). this task is crucial in understanding user opinions and sentiments in various applications such as customer feedback analysis, social media monitoring, and product reviews. surveys @xcite @xcite have extensively studied the evolution and trends in absa methodologies, particularly those employing deep learning techniques, attention mechanisms, and pre-trained language models (plms). their work highlights that adopting deep learning methods has significantly advanced research in asqp. nonetheless, a comprehensive overview of asqp and its solutions is lacking. @xcite provides the first comprehensive review of asqp, addressing gaps by reclassifying absa subtasks, summarizing various plm methods applied to asqp, and exploring chatgpt in sentiment analysis. the field has advanced significantly and is primarily driven by plm approaches. transformer-based models like bert (kenton and toutanova, 2019), bart @xcite , and t5 @xcite have set new benchmarks by leveraging pretrained language models (plms) and fine-tuning techniques to achieve state-of-the-art results. however, several challenges remain. bertbased methods @xcite , while pioneering the use of transformer models, face limitations when combined with traditional techniques like crf, leading to issues such as gradient vanishing and error propagation. bartbased approaches @xcite introduced encoder-decoder models but lack extensive fine-tuning research and often borrow methods from t5. although t5-based approaches have recently achieved promising results in the asqp field, research on implicit sentiment remains limited. moreover, most methods only research simple input and output template construction @xcite @xcite @xcite @xcite or basic contrastive learning @xcite . in this work, we enhance quad extraction accuracy for implicit aspects and opinions using the t5 model, improving asqp fine-tuning through deeper sentiment instruction and contrastive learning. implementing this approach faces several challenges, including designing instruction learning templates, constructing multi-dimensional contrastive learning to infer implicit sentiments from single-sentence datasets without additional context, and fine-tuning the t5 model to avoid overfitting or underfitting, thus enhancing accuracy and robustness. we propose a novel framework, itscl (instruction tuning and supervised contrastive learning), designed to improve accuracy and robustness, particularly for implicit aspects and opinions. the instruction tuning (it) component uses explicit instruction learning to provide detailed input and output constructions, guiding plms and indicating that aspects and opinions can be implicit. this custom template is designed explicitly for asqp task. the multi-dimensional supervised contrastive learning (scl) component employs four fully connected layers to optimize similarity for same-label representations and minimize it for different labels (sentiment, aspect, opinion, aspect&opinion). this approach aligns representations and integrates similarities and differences across dimensions, effectively capturing explicit and implicit sentiment. experiments show our approach significantly outperforms state-of-the-art methods on two benchmark datasets, excelling in explicit sentiment identification and implicit aspect and opinion detection. the key contributions of this paper are: ## conclusions in this paper, we introduced itscl, a unified framework designed to enhance the prediction accuracy of aspect sentiment quads, particularly focusing on implicit aspects and opinions. by combining instruction tuning through prompt engineering with highly aligned plm templates, itscl enables models to acquire knowledge more effectively and identify implicit sentiments. the contrastive learning framework also improves model performance by optimizing the similarity between representations with the same label and differentiating those with different labels. our extensive experiments on benchmark datasets demonstrate that itscl significantly outperforms existing methods.