Conditioning on Dialog Acts improves Empathy Style Transfer (2023)

Abstract: We explore the role of dialog acts in style transfer, specifically empathy style transfer – rewriting a sentence to make it more empathetic without changing its meaning. Specifically, we use two novel few-shot prompting strategies: target prompting, which only uses examples of the target style (unlike traditional prompting with source/target pairs), and dialog-act-conditioned prompting, which first estimates the dialog act of the source sentence and then makes it more empathetic using few-shot examples of the same dialog act. Our study yields two key findings: (1) Target prompting typically improves empathy more effectively while maintaining the same level of semantic similarity; (2) Dialog acts matter. Dialog-act-conditioned prompting enhances empathy while preserving both semantics and the dialog-act type. Different dialog acts benefit differently from different prompting methods, highlighting the need for further investigation of the role of dialog acts in style transfer.

## introduction expressing empathy is important for communication in applications ranging from customer service to mental health. although there is no universal definition of empathy, most definitions share a core of having the ability to understand and experience the feelings of others @xcite . 1 recent research has explored empathetic response generation, where the language model generates a response that conveys both the intended message and a sense of empathy given the input context @xcite @xcite . however, empathetic response generation is not applicable to more sensitive cases such as medicine and therapy, where the content of the responses requires the supervision 1 of course, empathic chatbots do not actually experience feelings; they merely speak as if they did. of medical experts. a key issue for state-of-the-art conversational models is hallucination @xcite , where the models provide fake or incorrect information in their re-sponses. physicians often (reasonably) do not trust the content of computer-generated responses but use large language models (llm) such as @xcite to rephrase their responses to express more empathy. this is style transfer: paraphrasing a sentence from one style to another without changing its content ( few-shot prompting, also called "in-context learning", has proven remarkably good at style transfer, giving performance competitive with more data-intensive fine-tuning @xcite . despite having some limitations @xcite , prompting is attractive in eliminating the need for extensive curated data. previous work compares zero-shot and pairwise few-shot prompting (where there are, for example, content-matched pairs of high and low empathy responses) on text style transfer @xcite . zero-shot prompting is prone to generating illogical or nonsensical outputs, while pairwise few-shot prompting requires optimal choices of sampled sentence pairs to maximize its performance @xcite . in this paper, we investigate empathy style transfer using prompt-based inference with gpt-4. we propose two sets of novel prompting approaches: target prompting and dialog-actconditioned prompting. target prompting is inspired by the difficulty of collecting counterparts of empathetic sentences, and uses sentences of only the target style as its examples (i.e., highly empathetic sentences). it contrasts with the more common pairwise prompting, where the prompts contain semantically matched pairs of high and low empathy responses. dialog-act-conditioned prompting is inspired by the fact that dialog acts play a crucial role in pragmatics and convey essential information about our cognitive processes, which are closely related to our empathy expressions @xcite . we propose two types of dialog-act-conditioned prompting: (a) selecting few-shot examples based on the dialog act of the source sentence and (b) explicitly specifying the dialog act in the prompt format in addition to (a). we test our prompting strategies and focus our evaluations on empathy transfer strength and content preservation. in summary, we find that: text style transfer one conventional way to do text style transfer was style-content disentanglement, where encoder-decoder structures such as variational autoencoders @xcite empathy content input sentence 2.74 yes i'm worried that things like that will continue to happen in the us also. zero-shot response 1.97 i understand that such events will persist in the us as well. few-shot target prompting response 2.84 i share your concern that such events may persist in the us, causing distress and unease among its people. few-shot da-target prompting response 3.23 i can genuinely understand the concern that events like these may persist in the us, causing a sense of unease and apprehension in our hearts. more recently, prompt-based inference with llms was significantly more efficient than finetuning, and it eliminated the issue of lack of data availability in text style transfer. @xcite used the prevalent zero-shot and few-shot prompting with gpt-3 @xcite for arbitrary style transfer. @xcite prompts the large language model multiple times to generate multiple responses and reranks them based on three evaluation metrics: transfer strength, semantic similarity, and fluency. however, to our knowledge, none of the text style transfer work has considered dialog acts so far. dialog act dialog act is an indicator of the intent or purpose of a speaker's utterance within a conversation. it has long been acknowledged that understanding dialog acts is crucial for discourse analysis, especially for pragmatic aspects like sentiments and emotions. @xcite first attempted to model the joint distribution of dialog acts and sentiments, and @xcite further enhanced the model and acknowledged the strong correlation between dialog acts and sentiments. later, @xcite incorporated emotions into dialog act classification and confirmed that emotions affect our dialog act choices; and @xcite was able to build neural annotators to jointly annotate emotions and dialog acts. more recently, @xcite found out that dialog act prediction significantly improved depression detection, hinting at the strong potential of dialog acts in understanding the emotional and mental aspects of discourse. dialog acts are rarely, if ever, used in text style transfer, but their use has enhanced performance in other natural language generation tasks @xcite @xcite . motivated by these papers, we believe that using dialog acts will benefit empathy style transfer. ## results target vs pairwise as shown in ## conclusion in this paper, we propose two novel few-shot prompting methods for empathy style transfer: target prompting and dialog-act-conditioned prompting. we examine and confirm the conditionality of empathy scores on the dialog acts of the source sentences. target prompting is better at increasing empathy than prompting with traditional source/target pairs. conditioning the few-shot prompt on the dialog act of the source sentence leads to stronger empathy transfer strength and better dialog act preservation. we expect that accounting for dialog acts will prove important for other style transfer tasks.