Progressive Tuning: Towards Generic Sentiment Abilities for Large Language Models (2024)

Abstract: Understanding sentiment is arguably an advanced and important capability of AI agents in the physical world. In previous works, many efforts have been devoted to individual sentiment subtasks, without considering interrelated sentiment knowledge among these subtasks. Although some recent works model multiple sentiment subtasks in a unified manner, they merely simply combine these subtasks without deeply exploring the hierarchical relationships among subtasks. In this paper, we introduce GSA-7B, an open-source large language model specific to the sentiment domain. Specifically, we deeply explore the hierarchical relationships between sentiment subtasks, proposing progressive sentiment reasoning benchmark and progressive task instructions. Subsequently, we use Llama2-7B as the backbone model and propose parameter-efficient progressive tuning paradigm which is implemented by constructing chain of LoRA, resulting in the creation of GSA-7B. Experimental results show that GSA7B as a unified model performs well across all datasets in the progressive sentiment reasoning benchmark. Additionally, under the few-shot setting, GSA-7B also exhibits good generalization ability for sentiment subtasks and datasets that were not encountered during its training phase.

heading: 1 Introduction
text: Sentiment plays a crucial role in social interaction. Minsky, in his book “Society of Mind” (Minsky, 1988), mentions that equipping machines with the ability to understand the sentiment across different scenarios has consistently been the steadfast goal of researchers. With Large Language Models (LLMs) playing a growing role in our lives, developing LLMs with sentiment intelligence could be better at communicating with us, collaborating with us, and understanding us (Gandhi et al., 2023; Shu et al., 2021).
† Corresponding author.
In prior literature, many efforts (Raffel et al., 2020; Zhao et al., 2022; Li et al., 2022; Hu et al., 2023; Hou et al., 2023; Qiao et al., 2023; Wang et al., 2023b) have been devoted to individual sentiment subtasks, such as Sentiment Analysis (SA), Emotion Recognition in Conversation (ERC), and Sarcasm Detection (SD). However, it has become increasingly clear that there is an interrelated sentiment knowledge among these subtasks. Therefore, integrating all subtasks into a unified model to enhance the sentiment understanding ability of the model has emerged as a significant objective. Although some recent works (Li et al., 2023; Hu et al., 2022; Shah et al., 2023) have emerged to model multiple sentiment subtasks in a unified manner, they merely simply combine various sentiment subtasks without delving into the hierarchical relationships among subtasks. Moreover, there is currently a lack of an open-source LLM with sentiment analysis capabilities that can perform well across multiple sentiment subtasks.
In this paper, we introduce an open-source LLM endowed with sentiment analysis capabilities. The complete process can be divided into three steps: (1) As shown in Figure 1, the task of SA requires reasoning based on the semantic information of sentences. The ERC task, building upon the SA task, further focuses on the contextual information of the target utterance. The SD task, building upon both SA and ERC tasks, additionally pays attention to whether the target utterance expression exhibits linguistic incongruity. We map the SA, ERC, and SD tasks to Level 1-3, respectively, forming a progressive sentiment reasoning benchmark. (2) Consistent with the perspective of progressive sentiment reasoning, we design progressive task instruction for SA, ERC, and SD tasks. (3) We propose parameter-efficient progressive tuning paradigm, which is implemented by constructing chain of LoRA (Hu et al., 2021b; Xia et al., 2024). Within this paradigm, fine-tuning for each
14392
level of task begins with the fine-tuned model from the previous level, leading to the residual learning between different level tasks. In Section 4.2, we provide a detailed analysis of the multiple advantages of the progressive tuning paradigm, including its effectiveness in mitigating the issue of catastrophic forgetting. We use Llama2-7B (Touvron et al., 2023) as the backbone model, and the finetuned model from the final level task is referred to as the result model, named GSA-7B.
The experimental results indicate that GSA-7B as a unified model achieves state-of-the-art (SOTA) results across all task types in the progressive sentiment reasoning benchmark and performs better on half of the datasets compared to SOTA models tailored to individual datasets. Additionally, in the few-shot scenario, GSA-7B demonstrates a commendable ability to generalize across sentiment subtasks and datasets that were not encountered during its training phase.
Our main contributions are summarized as follows:
• We deeply explore the hierarchical relationships between sentiment subtasks, proposing progressive sentiment reasoning benchmark that includes SA (Level 1), ERC (Level 2), and SD (Level 3) tasks.
• Consistent with the perspective of progressive
sentiment reasoning, we design progressive task instruction for SA, ERC, and SD tasks.
• We propose progressive tuning paradigm, which is implemented by constructing chain of LoRA, which can effectively mitigate the issue of catastrophic forgetting.
• We introduce GSA-7B, an open-source LLM specific to the sentiment domain. Experimental results show that GSA-7B as a unified model performs well across all datasets in the progressive sentiment reasoning benchmark and exhibits good generalization ability for sentiment subtasks and datasets not seen during training.

heading: 2 Background and Related Work

heading: 2.1 Low Rank Adaptation
text: The study by Hu et al. (2021b) focuses on improving the fine-tuning efficiency of LLMs by training considerably smaller low-rank decomposition matrices for specific weights. It posits that weight updates during task adaptation have a low "intrinsic rank" and introduce trainable low-rank decomposition matrices into each layer of the Transformer architecture. Consider a weight matrix Wpretrained from the pre-trained model, the weight update ∆W for task adaptation is represented with a low-rank
decomposition BA. The forward pass with LoRA is as follows:
Wpretrainedx+∆Wx = Wpretrainedx+BAx (1)
where Wpretrained,∆W ∈ Rd×k, A ∈ Rr×k, B ∈ Rd×r and r ≪ min(d, k). A is typically initialized with random Gaussian initialization and B is initialized with zero to have ∆W = 0 at the beginning of training. During training, Wpretrained is frozen and only B, A are optimized. During deployment, the learned low-rank matrices can be merged with the frozen weights of the pre-trained model.

heading: 2.2 Multi-task Unified Framework
text: LLMs as Backbone Text-based LLMs (Brown et al., 2020; Touvron et al., 2023; Chiang et al., 2023; Du et al., 2021) have demonstrated remarkable and even human-level performance in many NLP tasks (Achiam et al., 2023). Meanwhile, instruction tuning (Wei et al., 2021; Chung et al., 2022; Peng et al., 2023), where data is organized as pairs of instruction (or prompt) and response, has emerged as an LLM training paradigm. Building on this foundation, a significant amount of research has been dedicated to developing LLMs for specific domains, including the healthcare (Yang et al., 2023) domain, the education (Lee et al., 2024) domain, the law (Cui et al., 2023) domain, the finance (Zhang and Yang, 2023) domain, and so on.
PLMs as Backbone In the sentiment domain, Yan et al. (2021) employ an improved BART (Lewis et al., 2019) architecture to solve all ABSA subtasks in an end-to-end framework. Hu et al. (2022) propose a multimodal sentiment knowledgesharing framework that unifies MSA and ERC tasks from features, labels, and models. Shah et al. (2023) retrofit language models produce emotionaware text representations, applying to multiple sentiment subtasks. Liu et al. (2023) propose a quantum probability framework for joint sarcasm detection and sentiment analysis.

heading: 2.3 Sentiment Analysis, Emotion Recognition in Conversation and Sarcasm Detection
text: Sentiment Analysis The work in this field mainly focuses on fine-tuning pre-trained language models (such as RoBERTa (Liu et al., 2019), XLNet (Yang et al., 2019), and T5 (Raffel et al., 2020)) on specific datasets, achieving promising results.
Emotion Recognition in Conversation Unlike the basic SA task, ERC is a more practical endeavor that involves predicting the emotion label of each utterance based on the surrounding context. Existing works can be roughly divided into sequence-, graph-, and Transformer-based methods. Sequence-based methods: Hu et al. (2021a) propose a cognitive-inspired network that uses multiturn reasoning modules to capture implicit emotional clues in conversations. Zhao et al. (2022) utilize GRUs to fuse commonsense knowledge and capture complex interactions in the dialogue. Graph-based methods: Ghosal et al. (2019) treats the dialogue as a directed graph, where each utterance is connected with the surrounding utterances. Ishiwatari et al. (2020) introduces a positional encoding module to simultaneously consider speaker interactions and sequence information. Transformer-based methods: Shen et al. (2021) adopt a modified XLNet to deal with longer context and multi-party structures. Li et al. (2022) utilize supervised contrastive learning and a response generation task to enhance BART’s ability for ERC.
Sarcasm Detection The work in this field mainly focuses on capturing incongruity patterns. For example, Pan et al. (2020) utilizes deep neural networks augmented by attention mechanisms for explicitly exploring the contrast and incongruity on word-level or snippet-level. Wang et al. (2023b) introduce iterative incongruity graph structure learning to augment affective dependency graphs for sarcasm detection.

heading: 3 Progressive Sentiment Reasoning Benchmark
text: As shown in Figure 1, different sentiment subtasks form a progressive reasoning chain, leading to a renewed understanding of each sentiment subtask from a reasoning perspective. This section provides a description of the datasets that constitute the sentiment reasoning benchmark. More details can be found in Table 1.
Level 1: Sentiment Analysis SST-2 (Socher et al., 2013), MOSI (Zadeh et al., 2016) and MOSEI (Zadeh et al., 2016) are three widely used datasets for sentiment analysis focusing on sentiment polarity at the sentence (single-turn utterance) level. For the MOSI and MOSEI datasets, we only use their textual modal information.
Level 2: Emotion Recognition in Conversation IEMOCAP (Busso et al., 2008), MELD (Poria et al., 2018) and EmoryNLP (Zahiri and Choi, 2018) are three widely used datasets for emotion recognition in conversation, which aims to identify emotions conveyed in each utterance within the dialogue context (Poria et al., 2019). For the IEMOCAP and MELD datasets, we only use their textual modal information.
Level 3: Sarcasm Detection MUStARD (Castro et al., 2019) and MUStARD++ (Ray et al., 2022) are two widely used datasets for sarcasm detection in a conversational setting, which involves finding linguistic expression and emotional states incongruity. For the MUStARD and MUStARD++ datasets, we only use their textual modal information.

heading: 4 Progressive Tuning
text: In this section, we first introduce the task instruction used for progressive tuning, followed by a detailed description of the progressive tuning process.

heading: 4.1 Progressive Task Instruction
text: Consistent with the perspective of progressive sentiment reasoning, for basic SA tasks, we design instructions to focus on the semantic information of sentence (single-turn utterance):
Instruction for SA task
{sentence} Consider the semantic information of sentence to determine its sentiment polarity. {label list}
For the ERC task, we design instructions to focus on both the semantic information of the target
utterance and the contextual information of the dialogue it is situated in, to determine the emotion of the target utterance accurately:
Instruction for ERC task
{dialogue} {target utterance} Consider both the semantic information of the target utterance itself and the contextual information of the dialogue it is situated in. {label list}
For the SD task, we design instructions to focus on if there’s an incongruity between the emotional states conveyed by the target utterance’s semantics information and contextual information, and if the target utterance’s expression shows any linguistic incongruity:
Instruction for SD task
{dialogue} {target utterance} Consider whether there is an incongruity between the emotional states of the target utterance’s semantic information and its contextual information, and whether the expression of the target utterance involves linguistic incongruity. {label list}
Examples of progressive task instruction and a comparison of instruction for different level of tasks are shown in Figure 2 and Table 2.

heading: 4.2 Progressive Chain of LoRA
text: The key idea of our method is to form a chain of LoRA and progressively learn the low-rank adaptation LoRA modules. As illustrated in Figure 2, we first utilize the InstructionSA to fine-tune the Llama2-7B model on the SA task:
WSA = Wpretrained +∆WSA (2)
where Wpretrained ∈ Rd×k represents the pretrained weights matrix of the Llama2-7B model and ∆WSA = B1A1 represents the weight update occurred during fine-tuning. Then, starting from WSA, we continue to fine-tune on the ERC task using InstructionERC :
WERC = WSA +∆WERC
= Wpretrained +∆WSA +∆WERC (3)
where ∆WERC = B2A2. Similarly, starting from WERC , we continue to fine-tune on the SD task using InstructionSD:
WSD = WERC +∆WSD
= Wpretrained +∆WSA +∆WERC +∆WSD (4)
where ∆WSD = B3A3. Each low-rank tuple (Ai, Bi) is obtained by optimizing:
arg min BiAi
L (Wpretrained + i∑
j=1
BjAj) (5)
where L is the task-specific objective function. Within the progressive tuning paradigm, finetuning (A2, B2) can be viewed as learning the residual of WERC −WSA, which is not only an easier optimization problem compared to learning WERC from scratch but also can better coordinate semantic and contextual information for accurate emotion recognition in conversation. Furthermore, we think that the residual of WERC −WSA represents the learning of contextual information and a broader, more comprehensive understanding of semantic information. Our method ensures that during the fine-tuning process for the ERC (Level 2) task, attention is also given to the semantic information relevant to the SA (Level 1) task, effectively mitigating the problem of catastrophic forgetting. The same analytical perspective is also applicable to fine-tuning (A3, B3).
The fine-tuned model from the final level is referred to as the result model, named GSA-7B.
WGSA = WSD = Wpretrained + 3∑
j=1
BjAj (6)
where WGSA represents the weights of GSA-7B model.

heading: 5 Experimental Setup

heading: 5.1 Baselines
text: In this section, we report the unified model for sentiment subtasks and current SOTA models for each dataset in the progressive sentiment reasoning benchmark, which we use as baselines to compare with the performance of GSA-7B.
Unified Models KEA(Suresh and Ong, 2021) and RobertaEmo(Shah et al., 2023) retrofit language models produce emotion-aware text representations, applying to multiple sentiment subtasks. TLearn (Shah et al., 2023) is a transfer learning paradigm that initially fine-tunes on the emotion recognition task, followed by further fine-tuning on end tasks using a linear classification head. Quiet(Liu et al., 2023) is a quantum probability framework for joint sarcasm detection and sentiment analysis. UniMSE (Hu et al., 2022) is a sentiment knowledge-sharing framework that unifies sentiment analysis and emotion recognition in conversation tasks from features, labels, and models. UniSA (Li et al., 2023) unifies multiple sentiment subtasks under a single generative framework.
SOTA Models T5-11B (Raffel et al., 2020), serving as a pre-trained model, is fine-tuned on the SST-2 and achieves SOTA results. UniMSE is the current SOTA model for MOSI and MOSEI. InstructERC(Lei et al., 2023) reformulates the ERC task from a discriminative framework to a generative framework based on LLMs. It is the current SOTA model for IEMOCAP, MELD and EmoryNLP. MIL(Zhang et al., 2023) proposes a multi-task learning mechanism to capture correlations and differences across sarcasm detection and sentiment analysis tasks. It is the current SOTA model for MUStARD. Gaze(Tiwari et al., 2023) proposes the utilization of synthetic gaze data to improve the task performance for sarcasm detection. It is the current SOTA model for MUStARD++.

heading: 5.2 Evaluation Metrics
text: We use the same evaluation metrics as employed in the original paper of the dataset. Weighted Accuracy is used for SST-2, 2-category Accuracy is used for MOSI and MOSEI, and F1 scores are used for IEMOCAP, MELD, EmoryNLP, MUStARD and MUStARD++.

heading: 5.3 Implementation Details
text: We develop our approach using the PyTorch framework and the Transformers library (Wolf et al., 2020). We use Llama2-7B (Touvron et al., 2023) as our backbone model. In implementing LoRA, we adhere to the practice outlined in (Hu et al., 2021b), introducing trainable low-rank modules to the self-attention layer. We use a random Gaussian initialization for Ai and set Bi to zero, resulting in the value of BiAi being zero. We use Adam optimizer (Kingma and Ba, 2014) to update LoRA parameters and set the learning rate of {8e-5, 2e4, 1e-4} for progressive tuning of different level tasks. Greedy search is used during inference if not specified.

heading: 6 Result and Analysis

heading: 6.1 Main Results
text: We report the test performance of our method and baseline across all datasets in the progressive sentiment reasoning benchmark. The experimental results are shown in Table 3.
Compared to unified models, our method brings average absolute performance improvements of +0.28 , +3.34 , and +7.43 on SA, ERC, and SD tasks, respectively. Compared to SOTA models tailored to individual dataset, although our method can be viewed as a unified model, it brings absolute performance improvements of +0.39 , +0.64 , +2.11 , +1.43 on IEMOCAP (Level 2), EmoryNLP (Level 2), MUStARD (Level 3), and MUStARD++ (Level 3) datasets, respectively.

heading: 6.2 The Effect of Key Components
text: In this section, we analyze the knowledge transfer between tasks of different levels in the progressive sentiment reasoning benchmark and the effect of the progressive tuning paradigm. We analyze based on the model’s performance on the ERC task.
Knowledge Transfer To explore knowledge transfer between tasks of different levels, we per-
form LoRA fine-tuning on the Llama2 model using only three datasets from the ERC task. As shown in Table 4, compared to models fine-tuned on tasks across all levels, the models fine-tuned only on ERC tasks exhibited a noticeable average absolute performance decrease.
Progressive Tuning Paradigm Compared to multi-task tuning, progressive tuning has achieved better results across all datasets in the ERC task, as shown in Table 4. This represents that the progressive tuning strategy has established more accurate representations of semantic and contextual information for the target utterance in the ERC task, which also reflects its effective mitigation of the catastrophic forgetting problem to a certain extent.

heading: 6.3 Few-shot Generalization
text: Quantitative Analysis To demonstrate the generalizability of our method in sentiment subtasks:
(1) we conduct experiments on hateful detection (Basile et al., 2019) and emoji prediction (Barbieri et al., 2018) tasks, (2) we conduct experiments on sentiment analysis (Rosenthal et al., 2019) and emotion recognition (Mohammad et al., 2018) datasets beyond the progressive sentiment reasoning benchmark. All experiments are conducted under the few-shot setting.
The experimental results in Table 5 demonstrate that our method, under the few-shot setting, performs well on sentiment subtasks and datasets not included in the progressive sentiment reasoning benchmark. This result reveals that our method has learned general sentiment knowledge during the training phase.
Qualitative Analysis Rooting in the nature of LoRA, Wang et al. (2023a) propose an insight that
LoRA parameters are not mere numerical adjustments but encapsulate crucial directions for model gradient updates. Under the progressive tuning paradigm, fine-tuning for each level task starts with the model fine-tuned on the previous level task, learning the residuals between different level tasks. Furthermore, as illustrated in Figure 3, considering that various sentiment subtasks share similar gradient directions, when faced with a new sentiment subtask, the model can quickly and easily learn the patterns of the new task, thereby achieving competitive performance even in a few-shot setting.

heading: 6.4 Case Study
text: In Table 6, we exemplify two cases from the SA and SD tasks.
In the first case, we employ the model fine-tuned on the SA task to determine that the sentiment polarity of the sentence "I mean I don’t regret seeing
it." as incorrect, whereas the GSA-7B model provides the correct answer. We believe that this is due to during the fine-tuning process for the ERC (Level 2) and SD (Level 3) task, attention is also given to the semantic information relevant to the SA (Level 1) task, leading to a broader, more comprehensive understanding of semantic information, effectively mitigating the problem of catastrophic forgetting.
In the second case, we compare the result of multi-task tuning and the GSA-7B model on an instance of SD task. Due to the presence of both ’good’ and ’bad’ words in the target utterance, the multi-task tuning model classified it as sarcastic. For the GSA-7B model obtained by our progressive fine-tuning, it not only focuses on the linguistic expression of the target utterance but also takes into full consideration the emotional state corresponding to the contextual and semantic information, possessing a better cognitive pattern.

heading: 7 Conclusion
text: In this paper, we deeply explore the hierarchical relationships between sentiment subtasks, proposing progressive reasoning benchmark that includes SA (Level 1), ERC (Level 2), and SD (Level 3) tasks and progressive task instruction. Subsequently, we propose parameter-efficient progressive tuning paradigm to finetune the Llama2-7B model. The resulting model is named GSA-7B, a specialized open-source LLM for the sentiment domain. Experimental results show that GSA-7B performs well across all datasets in the progressive sentiment reasoning benchmark and exhibits good generalization ability for sentiment subtasks and datasets that were not encountered during its training phase, which reveals the feasibility of building AI that can understand sentiment.
Limitations
There are two major limitations in this study. Firstly, there are relatively few task types in the progressive sentiment reasoning benchmark, which can be further enriched by conducting more finegrained and in-depth exploration of tasks in the sentiment domain. For example, incorporating generative tasks such as empathetic responses into the benchmark. Secondly, in constructing the progressive sentiment reasoning benchmark, we only utilize the information from the textual modality in each dataset. Towards sentiment analysis capabilities for Multimodal LLM is also important, which we treat as future work.

heading: Acknowledgements
text: This work is supported by the National Natural Science Foundation of China (No. 62376245), the National Natural Science Foundation of Zhejiang Province (LY24C090001) and the Fundamental Research Funds for the Central Universities.

heading: Future Work
text: Towards sentiment analysis capabilities for Multimodal LLM is also important, which we treat as future work