Improving In-Context Learning with Prediction Feedback for Sentiment Analysis (2024)

Abstract: Large language models (LLMs) have achieved promising results in sentiment analysis through the in-context learning (ICL) paradigm. However, their ability to distinguish subtle sentiments still remains a challenge. Inspired by the human ability to adjust understanding via feedback, this paper enhances ICL by incorporating prior predictions and feedback, aiming to rectify sentiment misinterpretation of LLMs. Specifically, the proposed framework consists of three steps: (1) acquiring prior predictions of LLMs, (2) devising predictive feedback based on correctness, and (3) leveraging a feedbackdriven prompt to refine sentiment understanding. Experimental results across nine sentiment analysis datasets demonstrate the superiority of our framework over conventional ICL methods, with an average F1 improvement of 5.95%.1

heading: 1 Introduction
text: Sentiment analysis aims to detect subjective opinions within texts automatically (Medhat et al., 2014), covering tasks such as sentiment classification, aspect-based sentiment analysis, and emotion detection (Zhang et al., 2018).
Previous studies proposed many supervised methods for sentiment analysis (Xu et al., 2019; Li et al., 2021a). To avoid their reliance on large amounts of human-annotated data, some studies attempted to use limited data to recognize sentiment yet obtained mediocre results. With the advent of large language models (LLMs) (Brown et al., 2020; Touvron et al., 2023a), studies have revealed that LLMs can yield promising performance on sentiment analysis via in-context learning (ICL) paradigm (Li et al., 2023; Wang et al., 2023), which utilizes only few-shot input-label pairs selected from a candidate example pool.
∗ Corresponding Author 1The source code for our framework is available at https:
//github.com/HITSZ-HLT/Feedback-ICL.
Despite achieving favorable results, the conventional ICL paradigm still faces a concerning limitation. Namely, through the provided examples, LLMs fail to differentiate subtly similar sentiments effectively. Consequently, they would predict plausible yet incorrect sentiment labels. As depicted in Figure 1a, although LLMs can clearly distinguish between positive and negative polarities, they often mistakenly categorize neutral into others. In addition, as shown in Figure 1b, LLMs frequently mislabel fine-grained sentiments as relevant but wrong labels, such as joy and optimism, stemming from their incapacity to understand nuanced sentiments with similar contexts.
Inspired by the human learning process, where individuals initially make plans based on prior knowledge and adjust their understanding through actual feedback (Bélanger, 2011), we propose to integrate feedback on prior predictions into ICL, aiming to rectify sentiment misunderstandings of LLMs. Specifically, our framework first yields prior predictions for each candidate example using traditional ICL. We then categorize examples into two sub-pools based on correctness and exploit feedback to illustrate differences between prior predictions and human annotations. Finally, during inferring, we select relevant examples from each subpool and use a specific feedback-driven prompt to
3879
wrap input, prediction, label, and feedback. Unlike conventional ICL, where LLMs only see correct labels, our framework effectively directs LLMs to adjust their sentiment understanding and reasoning to align more closely with label perception through prediction and feedback.
Experimental results on nine sentiment analysis datasets show that our framework outperforms existing ICL baselines by 5.95% in average F1. Further discussions indicate its effectiveness and robustness. Moreover, the framework also yields competitive results when extended to other tasks.

heading: 2 Preliminary
text: Sentiment analysis aims to predict the sentiment label y′ of an input text x. Here, different tasks may have different label spaces C and inputs.2 In ICL paradigm, given an input x and k-shot incontext examples {(xi, yi)}ki=1 retrieved from a pre-defined candidate pool P (its size is relatively small), a frozen LLM M is used to predict y′.
y′ = argmax y∈C M(y|(x1, y1), ...(xk, yk), x) (1)
where we ignore the task instruction and example template for simplicity. To avoid irrelevant outputs, we employ a constrained decoding strategy that ensures only label words within C can be generated. Besides, we directly use these label words as the verbalizer for each class in classification.3

heading: 3 The Proposed Framework
text: As shown in Figure 2, our framework consists of three steps: 1) prior prediction acquisition, 2) pre-
2For example, aspect sentiment classification task needs to consider the effect of aspects in inputs (Pontiki et al., 2014).
3If a label word is split into subtokens, we use only the first subword for prediction, such as ’optim’ for ’optimism’.
dictive feedback design, and 3) test sample inference. Below is a detailed description.
Step 1: Prior Prediction Acquisition. This step focuses on acquiring the prior prediction y′i on each candidate example xi for subsequent feedback provision. To this end, examples from P are treated as inference targets. Following the traditional ICL, we randomly select other four input-label pairs from the candidate pool as demonstrations,4 which are combined with the task instruction to prompt the LLM for predictions (see Appendix A for more information). We refer to these predictions as prior predictions because they serve to reflect the prior sentiment understanding of LLMs.
Step 2: Predictive Feedback Design. The correctness of the prior predictions directly indicates whether LLMs can accurately grasp the sentiment of the corresponding examples. To elicit selfadjustments of LLMs in understanding and reasoning, we first classify the examples into two subpools, Pc and Pw, where the former includes correctly classified examples, and the latter contains wrong ones. We then provide each sub-pool with feedback in the natural language form:
feedback on Pc: You are correct! Stay determined and keep moving forward.
feedback on Pw: You are wrong! Make sure your prediction is accurate.
Step 3: Test Sample Inference. To complete the inference for the given test input, we first retrieve k/2 examples from each candidate sub-pool. Since our framework is retrieval-mode agnostic, any example retrieval technique can be employed here. In addition, we develop a feedback-driven prompt
4The reason for selecting four is to strike a trade-off between contextual richness and computational efficiency.
template to wrap the input, prediction, label, and feedback of each selected example into a quadruple. Subsequently, these quadruples are organized by Pw examples before Pc ones and sorted by descending relevance. Finally, the test input is set in the standard example template, with the label position left blank for prediction.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Datasets. We conduct experiments across three sentiment analysis tasks using nine distinct datasets, including Sentiment Classification (SC): SST2 (Socher et al., 2013), TwSenti (Rosenthal et al., 2017), Poem (Sheng and Uthus, 2020), and Finance (Malo et al., 2014); Aspect Sentiment Classification (ASC): Rest and Laptop (Pontiki et al., 2014), and Twitter (Dong et al., 2014); Emotion Detection (ED): EmoC (Chatterjee et al., 2019) and TwEmo (Barbieri et al., 2020). Detailed statistics are listed in Appendix B.1.
Baselines. To evaluate the effectiveness of the proposed framework, we combine it with various training-free example retrieval baselines for comparison, including Random, BM25 (Robertson et al., 2009), SBERT (Reimers and Gurevych, 2019), MMR (Ye et al., 2023), and KMeans (Zhang et al., 2023). Furthermore, we introduce BERT-FT, where the BERT-base model is fine-tuned directly on candidate pool examples. See Appendix B.2 for their specific settings.
Implementation Details. In this study, we utilize Llama-2 13B Chat (Touvron et al., 2023b) as the backbone LLM due to its moderate scale and
excellent ICL performance. Unless otherwise mentioned, we set the number of in-context examples to 4. The candidate pool is formed by sampling 300 label-balanced examples from each training set. We experiment with 3 different random seeds and present the average results. More implementation details are shown in Appendix B.3.

heading: 4.2 Main Results
text: Results shown in Table 1 indicate that our framework substantially enhances baseline performance on nearly all datasets. For example, augmenting K-means with our framework results in an average F1 increase of 5.91%, exhibiting its superiority. Meanwhile, compared with BERT-FT, our approach demonstrates outstanding performance on the majority of datasets, highlighting its efficacy in resource-limited scenarios.
Additionally, our framework notably excels in ED, where detecting subtly similar sentiments is crucial. Comparatively, ASC involves more complex aspect-based contextual understanding, constraining the improvement in these tasks.
Contrary to previous studies (Rubin et al., 2022; Li et al., 2023), we find that semantic similarity retrievals like SBERT negatively impact the performance. We suppose it is due to the demonstration bias when solving simple sentiment analysis tasks (Fan et al., 2023) and the lack of example complementarity (Ye et al., 2023).

heading: 4.3 Ablation Study
text: We perform the ablation study to explore the effect of each component, as presented in Table 2. When removing task instructions, we see performance
drops except for the Rest dataset, indicating its insensitivity to instructions with information-rich inputs. Additionally, both the removal of labels and prior predictions cause a notable decline, by averages of 10.13% and 7.92% respectively, highlighting the significance of their combination in our framework. Besides, employing alternative prediction sources or excluding feedback also leads to a slight decrease.

heading: 4.4 Effect on Subtle Sentiments
text: To demonstrate the impact of our framework on subtle similar sentiments, we visualize the prediction distributions as depicted in Figure 3. We can observe an obvious change of distribution in neutral, whose correct rate increases by 32%, while the other two categories are relatively stable. These results suggest that integrating predictive feedback could make more accurate distinctions between subtly similar sentiments.
4.5 Discussions5
Language Model Generalization. To evaluate the adaptability of our framework, we conduct model generalization experiments with various
5We present more analyses in Appendix D, including The Sensitivity of Feedback Prompt, Impact of the Number of Examples, and Impact of the Order of Examples.
LLMs. Specifically, we select three capable and prominent models: Mistral 7B Instruct (Jiang et al., 2023), GPT-3.5 Turbo (Ouyang et al., 2022), and GPT-4 (OpenAI, 2023).6 As illustrated in Table 3, incorporating our framework consistently enhances the performance of ICL, particularly with GPT3.5 Turbo, where the average F1 improvement is 4.72%, illustrating its generalizability. Furthermore, employing LLMs with more advanced comprehension such as GPT-4 significantly improves sentiment analysis results.
Impact of the Ratio of Error Examples. To assess the effect of the erroneous examples, we fix k at 8 and vary the number of examples selected from Pw, where the ratio is adjusted in 1/4 increments, ranging from 0 to 1. The results are depicted in Figure 4. We find that the framework underperforms with either no error examples or an excess of them. Meanwhile, as the quantity of incorrect examples increases, the performance initially rises and then declines, indicating that a relative balance of incorrect to correct examples is beneficial.
6The exact versions of the three models are as follows: Mistral-7B-Instruct-v0.2, gpt-3.5-turbo-0301, and gpt-4-0613.
Task Generalization. To demonstrate that our framework is not confined to adjusting sentiment understanding of LLMs, we conduct experiments on three additional datasets: P-Stance (Li et al., 2021b) for stance detection, TwIrony (Van Hee et al., 2018) for irony detection, and MNLI (Wang et al., 2019) for natural language inference (NLI). Results are illustrated in Table 5. Notably, the proposed framework also yields significant improvements on these datasets, with average F1 increasing by 1.85% for P-Stance, 3.88% for TwIrony, and 5.35% for MNLI. These results suggest that the adaptability of our prediction feedback mechanism can extend to a broader scope of language understanding tasks.
Case Study. To gain a deeper insight into the advantages of our framework, we perform the case study focusing on the outputs and explanations of the LLM, as detailed in Table 4. In Case 1, our framework identifies the emotion as joy rather than
the plausible yet incorrect optimism, and offers a more fitting explanation that aligns with the implied emotion. In Case 2, our framework correctly identifies the neutral polarity and avoids inaccurate sentiment interpretation. These cases reveal that our framework promotes self-adjustment of the LLM in sentiment analysis, refining both output and reasoning accuracy.

heading: 5 Conclusion
text: In this paper, we propose a novel ICL framework that utilizes prediction feedback akin to human learning. It improves ICL by incorporating both prior predictions and corresponding feedback into examples, tackling the difficulties LLMs encounter when identifying subtle sentiments. Experiments across various datasets confirm the advantages of our framework compared to conventional ICL, as well as its potential for broader applications.

heading: Acknowledgments
text: This work was partially supported by the National Natural Science Foundation of China 62176076, Natural Science Foundation of GuangDong 2023A1515012922, the Shenzhen Foundational Research Funding JCYJ20220818102415032 and JCYJ20210324115614039, the Major Key Project of PCL2021A06, Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies 2022B1212010005.

heading: Limitations
text: While our research significantly enhances the performance of conventional ICL and provides indepth analyses about the adjustment for sentiment understanding of LLMs, the inner mechanisms of the framework remain elusive due to the black-box nature of language models. Besides, our research primarily focuses on sentiment analysis and text classification within NLU, leaving more complex areas like text summarization and commonsense generation unexplored. We aim to broaden the scope of our framework in future work, exploring more insights and wider applicability.

heading: A Prompt Design
text: We present the task instructions and prompt templates utilized in our framework for each task in Table 14. Besides, for conventional ICL, the examples and test input are wrapped in the template that removes prediction and feedback, and the formatting word "Correct Label:" is replaced by "Label:".
We divide the feedback prompt into two parts, namely, feedback on correctness and feedback for analysis, abbreviated as FC and FA. Two manually designed feedback prompts are illustrated in Table 13 for further discussion (see D.1).
To generate explanations in the case study, we construct instructive forms7 and employ prompts: Provide the correct label for the following sample and explain your answer based on the above examples (and feedback).

heading: B Detailed Settings of Experiments

heading: B.1 Dataset and Metrics
text: We provide detailed statistics of each investigated dataset in Table 11. When establishing the candidate pool, we select instances only from the training set. Additionally, for Finance, lacking a standard split, we randomly allocate 20% of total samples to both the dev and test sets. For MNLI that does not offer publicly available test set labels, we evaluate by the dev set.
Across all sentiment analysis datasets utilized in this study, we uniformly apply two metrics for evaluation: Accuracy (Acc) and F1 score (F1). Moreover, we use binary-F1 for binary classification tasks and macro-F1 for all others.
7https://github.com/huggingface/blog/blob/ main/llama2.md#how-to-prompt-llama-2

heading: B.2 Baseline Details
text: (1) Random randomly selects k-shot examples from the candidate pool for each test sample. (2) BM25 (Robertson et al., 2009) assesses relevance through keyword overlap and sentence length, used by (Agrawal et al., 2023). (3) SBERT (Reimers and Gurevych, 2019) is a semantic-based retrieval method, where we use “paraphrase-mpnet-basev2” following (Li et al., 2023). (4) MMR (Ye et al., 2023) leverages BERTScore (Zhang et al., 2019) with maximal-marginal relevance for complementary example selection. (5) K-Means (Zhang et al., 2023) performs kmeans clustering to divide each dataset into four clusters. We then select examples randomly from each cluster. (6) BERT-FT (Devlin et al., 2019) fine-tunes “bertbase-uncased” using the candidate pool examples.

heading: B.3 More Implementation Details
text: Due to limited computational resources, test samples are restricted to 2,000 across the tasks: TwSenti, EmoC, P-Stance, and MNLI. Additionally, to accelerate inference, we load LLMs in fp16 precision. All experiments are conducted with an NVIDIA RTX A6000 GPU.

heading: C Additional Results

heading: C.1 Main Results in Accuracy
text: For a more comprehensive comparison with the performance of baseline methods, we show additional main results in accuracy, as shown in Table 12.

heading: C.2 Ablation Results on More Baselines
text: To comprehensively analyze the significance of each component within our framework, we conduct more ablation studies on two competitive baselines: Random and BM25. We report the results in Tables 6 and 7, respectively.

heading: C.3 Effect on Subtle Sentiments for More Datasets
text: To further illustrate how our framework corrects subtle sentiment understanding of the LLM and aligns predictions more closely with true labels, we visualize the improved prediction distributions on more datasets. The results are shown in Figure 5.
Inst Label Pred Feed Poem Rest TwEmo
✓ ✓ ✓ ✓ 61.27 71.76 62.88
✗ ✓ ✓ ✓ 53.86 71.73 61.80 ✓ ✗ ✓ ✓ 50.42 67.74 51.95 ✓ ✓ ✗ ✓ 52.45 70.05 50.32 ✓ ✓ R ✓ 60.07 70.18 62.64 ✓ ✓ Z ✓ 61.09 69.82 62.02 ✓ ✓ ✓ ✗ 54.68 70.84 62.03
(a) Normalized confusion matrices for the Rest dataset: BM25 (left) and BM25+Ours (right).
(b) Normalized confusion matrices for the TwEmo dataset: BM25 (left) and BM25+Ours (right).
(c) Normalized confusion matrices for the EmoC dataset: KMeans (left) and K-Means+Ours (right).
Figure 5: Effect on subtle sentiments for other datasets.

heading: C.4 More Case Studies
text: Additional cases are illustrated in Table 9. Observations reveal that incorporating prior predictions and feedback into examples not only corrects the sentiment labels but also yields more aligned explanations with human understandings.

heading: D More Discussions

heading: D.1 The Sensitivity of Feedback Prompt
text: To investigate the sensitivity of feedback design, we leverage the two hand-crafted feedback prompts shown in Table 13 and experiment with four combinations. As presented in Table 8, we find that on both datasets, the average variance in performance is within a 1% margin. These results indicate that the performance is not highly sensitive to different feedback prompts.
D.2 Impact of the Number of Examples
To analyze the effect of the example quantity, we perform experiments varying the number of examples (k = 2, 4, 6, 8, 12), as depicted in Figure 6. First, we observe a consistent progressive trend of F1 as the number of examples increases. Second, our method generally yields significant performance gains on both datasets compared to the baselines, except for the 12-shot scenario on TwEmo. This suggests the proposed framework can positively influence the LLM to understand and analyze sentiment within an optimal context length.
D.3 Impact of the Order of Examples To investigate the impact of example ordering, we first categorize three strategies: prioritizing wrong examples (wrong first), prioritizing correct examples (correct first), and alternating between the two. We then apply both ascending and descending arrangements based on retrieval scores. On this basis, we experiment with five extra permutations, as shown in Table 10. We find that the performance of Rest remains stable regardless of permutations, with a minor standard deviation of 0.51 in F1. Conversely, on TwEmo, descending ordering generally outperforms ascending ones. These findings suggest that although our framework is robust against the variability of strategy, the consideration of specific arrangement methods could be important.

heading: Future Work
text: We aim to broaden the scope of our framework in future work, exploring more insights and wider applicability