Emotion and Sentiment Guided Paraphrasing (2023)

Abstract: Paraphrase generation, a.k.a. paraphrasing, is a common and important task in natural language processing. Emotional paraphrasing, which changes the emotion embodied in a piece of text while preserving its meaning, has many potential applications, including moderating online dialogues and preventing cyberbullying. We introduce a new task of fine-grained emotional paraphrasing along emotion gradients, that is, altering the emotional intensities of the paraphrases in fine-grained settings following smooth variations in affective dimensions while preserving the meaning of the original text. We reconstruct several widely used paraphrasing datasets by augmenting the input and target texts with their fine-grained emotion labels. Then, we propose a framework for emotion and sentiment guided paraphrasing by leveraging pre-trained language models for conditioned text generation. Extensive evaluation of the fine-tuned models suggests that including fine-grained emotion labels in the paraphrase task significantly improves the likelihood of obtaining high-quality paraphrases that reflect the desired emotions while achieving consistently better scores in paraphrase metrics such as BLEU, ROUGE, and METEOR.

## introduction with the rise of social media and online chat rooms, the textual aspect of language is often found to be the only aspect of communication that is transferred over the internet. devoid of any intonations or accompanying facial movements, it is more challenging for people to decipher the true meaning and underlying emotion that a message is intended to convey, especially if that message incorporates the more complex aspects of speech. this could lead to negative social consequences. for example, political tweets from prominent figures without careful consideration can lead to political radicalization and conflicts. furthermore, on messaging apps such as discord, cyberbullies attack others with emotion-ladened words while innocent people send unnecessarily emotional messages in the heat of the moment. emotional paraphrasing could be an important solution to overly intense emotions expressed on social media @xcite and provide support toward moderation of hate speech @xcite . paraphrase generation (a.k.a. paraphrasing), a key task in natural language processing, involves generating an output text that preserves the meanings of the input text while including variations in words and grammars. the refined task of emotional paraphrasing has garnered much recent attention @xcite . its goal is to alter the underlying emotion associated with a sentence while maintaining its meaning. in this paper, we introduce a new task of finegrained emotional paraphrasing along emotion gradients, i.e., altering emotional intensities in fine grain following smooth variations in affective dimensions (e.g., from anger to annoyance) while preserving the overall meaning. first, we analyze and reconstruct existing paraphrasing datasets to adapt them for the current task. next, we propose the concept of an emotion-transition graph where transitions are based on the fine-grained emotions and their emotion gradients as identified by goemotions @xcite , and are constrained by specific goals of emotion transition. then, we develop a framework for emotion and sentiment guided paraphrasing by leveraging several pretrained language models for conditioned text generation under zero-shot, few-shot, and fully supervised settings. lastly, we conduct a comprehensive evaluation of the proposed framework with several datasets using metrics pertaining to both paraphrasing and emotion transition. ## related work this section discusses two main threads of related work: emotion classification and paraphrasing. ## experiments figure 3 illustrates the workflow of our experiments on preparing the train/test datasets for fine-grained emotional paraphrasing, conducting fine-tuning on various language models, and evaluating the emotional paraphrasing performance of these models. ## results and discussions in few-shot fine-tuning, we experimented with different limits for the numbers of text pairs following each fine-grained emotion transition in the train/test sets, 4/1, 8/2, 12/3, 16/4, and 20/5 per the 80/20 split. all few-shot train/test sets delivered better emotional transition performance than zeroshot and their paraphrasing performance became consistently better with 12/3 split and above. one important takeaway from the results is the similarity in performance of using sentiment ranges or fine-grained emotions as part of the prefix prompt to the models. we noticed that there was an insignificant difference in both the emotiontransition and paraphrasing performances of the two prefix types. an explanation for this behavior in the fine-tuned models may be that emotion transitions largely follow continuous gradients among emotions along certain affective dimensions and, therefore, lowering the sentiment intensity from an emotion often transitions to a same target emotion. this means that although the prefixes are different, the models learn the same emotion transitions that are embodied in the paraphrase pairs. figure 4 illustrates the success rates of t5 in transitioning texts between different sentiment intensity levels under different fine-tuning settings. we observe that fully supervised and few-shot finetuning both outperform zero-shot significantly in all sentiment intensity lowering transitions. fully supervised seems to perform better in emotion transitions lowering positive sentiments while few-shot better in lowering negative sentiments. importantly, we also observe that lowering from high positive or negative to low positive or negative is more challenging for the model than lowering to neutral level. 6 case study on transition graph ## conclusions and future work in this paper, we introduced a new task of finegrained emotional paraphrasing along emotion gradients. we developed a workflow for addressing this task by fine-tuning pre-trained language models under multi-task learning framework. our experiments have demonstrated that fine-tuned models perform significantly better than baseline models in both emotion transition and paraphrasing. for future work, there is still much to improve for fine-grained emotional paraphrasing. we will pursue better datasets for emotional fine-tuning or even develop new datasets for this purpose. we will further develop our approach on top of the state-ofthe-art large language models, e.g., gpt-4. we will also investigate more customized models beyond the baseline language models. for evaluation, we plan to conduct human studies as appropriate.