Aligning Black-Box LLM s for Aspect Sentiment Quad Prediction (2025)

Abstract: Aspect-Based Sentiment Analysis (ABSA) focuses on extracting opinions about specific aspects, with Aspect Sentiment Quad Prediction (ASQP) being the most complex sub-task. Large language models (LLMs) like GPT4 exhibit strong generalization yet struggle with ASQP due to a lack of task-specific alignment. Supervised small language models (SLMs), while effective in capturing task-specific patterns, lack the extensive knowledge of LLMs. To address this, we propose a framework that combines SLMs and LLMs using supervised in-context learning to align LLM outputs with human preferences. One SLM is supervised to generate candidate answers and guide LLMs with task-specific instructions, while another SLM acts as a reward model iteratively evaluates and refines LLM outputs. Experiments show that our framework significantly improves ASQP performance, demonstrating robustness, scalability, and potential for advancing alignment techniques in sentiment analysis.

## introduction aspect-based sentiment analysis (absa) is a finegrained sentiment analysis task that aims to extract opinions expressed toward specific aspects of a given target @xcite . among its subtasks, aspect sentiment quad prediction (asqp) represents the most challenging task, requiring the identification of aspect-category-opinion-sentiment quads from the text @xcite . common methods for solving asqp often rely on structured extraction techniques @xcite @xcite @xcite . however, with the rise of large language models (llms) such as chatgpt @xcite and claude @xcite , there is increasing interest in leveraging their strong generalization capabilities for asqp. these llms have demonstrated remarkable performance across diverse applications @xcite , but directly applying them to asqp remains challenge @xcite . one common approach to align language models with human preferences is supervised fine-tuning. while effective, fine-tuning huge llms (e.g.,gpt-4, claude) is infeasible for asqp due to their blackbox nature and the prohibitive computational costs of updating such massive models. alternatively, incontext learning @xcite has emerged as a practical strategy to guide black-box llms for downstream tasks. as shown in figure 1 (a), llms can follow instructions and generate outputs in forms that humans prefer. however, their answers often differ from what humans consider correct. this difference appears because llms rely on their pretrained knowledge, which may not include the spe-cific details required for asqp. as a result, their outputs tend to be biased or incomplete. in contrast, as shown in figure 1 (b), supervised slms can learn patterns that align well with human preferences for asqp. yet, slms have limited world knowledge, so their single-pass answers may still be incorrect or incomplete. surprisingly, we observe that by increasing the number of samples from one to ten, the probability of including a correct answer grows substantially, ultimately boosting the f1-score by more than 10% (see section 3.2). based on these observations, we propose a framework that combines the strengths of supervised slms and black-box llms to address asqp. specifically, we use slms to learn human preferences and transfer them to llms through incontext learning. we firstly supervise fine-tuning a slm to learn from human-annotated data and generates candidate answers during testing. by combining these candidate answers with well-designed instructions, we use the rich human supervision signals to guide llms toward aligning their outputs with human intent. secondly, we supervised fine-tuning another slm to act as a reward model to evaluate whether the llms' outputs align with human preferences @xcite . in particular, the llm can perform multiple rounds of sampling, and the reward model evaluates the correctness of its outputs, adding correct answers to the candidate answer list. by iteratively repeating this process, the llm can be effectively aligned with human preferences and progressively improve its ability to generate humandesired aspect sentiment quad predictions. we conduct extensive experiments to evaluate the proposed framework across various dimensions. our results reveal that directly applying in-context learning with black-box llms struggles to generate human-aligned asqp answers, while supervised fine-tuning of slms effectively captures human preferences. furthermore, integrating supervised signals of slm into context for guiding blackbox llms demonstrates significant improvements without requiring additional training data. comparative studies highlight the advantages of our candidate answer strategy and iterative alignment approach. the main contributions of this work can be summarized as follows: ## methods in this section, we first introduce the aspect sentiment quad prediction problem definition, then quantitatively analyze the zero-shot black-box llms compared with supervised slm. finally, based on the insights of the analysis, we explore aligning the black-box llms through supervised and reinforcement-enhanced context for aspect sentiment quad prediction as shown in figure 2(b) . ## experiments in this section, we introduce our experimental setup and implementation details, present our frameworks' performance on several standard datasets compared to competitive baselines. ## analysis and discussion the experiments showed that while qwen2.5 had a top@1 performance similar to scorer-gas, its larger gap between top@1 and top@10 led to better results within our framework. moreover, leveraging peft techniques like lora @xcite and well-deigned inference framework like vllm @xcite , it can achieve t5-largelevel gpu efficiency. therefore, we consider it a more promising slm backbone and select it for further analysis in this section. ## conclusion we proposes a framework that integrates supervised slms with black-box llms to address the challenges of aspect sentiment quad prediction. motivated by the complementary strengths of slms in capturing task-specific knowledge and llms in generalization, we designed a framework to align llm outputs with human preferences through in-context learning and iterative refinement. experimental results demonstrate that our framework significantly improves aspect sentiment quad prediction performance compared with incontext learning and supervised learning methods. in the future, we will explore extending this alignment framework to other fine-grained sentiment analysis tasks and further enhancing its adaptability to diverse datasets and tasks.