E mo N ews: A Spoken Dialogue System for Expressive News Conversations (2025)

Abstract: We develop a task-oriented spoken dialogue system (SDS) that regulates emotional speech based on contextual cues to enable more empathetic news conversations. Despite advancements in emotional text-to-speech (TTS) techniques, task-oriented emotional SDSs remain underexplored due to the compartmentalized nature of SDS and emotional TTS research, as well as the lack of standardized evaluation metrics for social goals. We address these challenges by developing an emotional SDS for news conversations that utilizes a large language model (LLM)-based sentiment analyzer to identify appropriate emotions and PromptTTS to synthesize context-appropriate emotional speech. We also propose subjective evaluation scale for emotional SDSs and judge the emotion regulation performance of the proposed and baseline systems. Experiments showed that our emotional SDS outperformed a baseline system in terms of the emotion regulation and engagement. These results suggest the critical role of speech emotion for more engaging conversations. All our source code is open-sourced.

## introduction in this work, we develop a task-oriented sds that can regulate emotional tts based on contextual cues (emotional sds) to enable more empathetic news conversations. task-oriented sdss must balance task-and social-goals to create engaging interactions @xcite , with emotional speech regulation being crucial among social-goals. for instance, synthesizing "sad" speech for tragic earthquake news can foster user empathy and engagement. appropriately managing emotional tone can thus enhance both user perception and overall experience @xcite . to support such needs, the field of affective computing has developed emotional tts techniques, which generate emotionally expressive speech by adjusting acoustic features like cadence, intensity, and pitch. recent emotional tts systems have achieved high-quality oral emotional expressions @xcite @xcite . however, despite these advances, taskoriented emotional sdss remain underexplored. this is primarily because socio-conversational research has been compartmentalized @xcite , with sds and emotional tts developing separately and lacking an integrated framework. moreover, evaluating social-goals like emotional speech regulation is difficult @xcite , as these goals are multidimensional and lack clear definitions, leading to few established evaluation metrics. thus, the gap between emotional tts capabilities and their effective integration into sds highlights an important area for further research. we develop a task-oriented emotional sds and propose its evaluation method. specifically, we focus on news summarization and q&a as a target task due to its extensive prior studies. for emotional speech regulation, we adopt a cascade sds architecture. we employ a prompttts @xcite fine-tuned on the esd dataset @xcite as our emotional tts model. for evaluation, we use an empathy scale originally @xcite and assess the sds's ability to regulate emotional speech. additionally, we manually evaluate both the system's emotional speech regulation and task achievement @xcite , comparing it with sdss that employ non-emotional tts. through this study, we contribute to the studies on socio-conversational system by: (i) providing a method for developing emotional sds; and (ii) proposesing an evaluation method of emotional sds. ## system design and method as depicted in figure 1 , we develop an emotional sds for news conversations using a cascade architecture, building on a strong baseline @xcite by adding emotional awareness via sentiment-guided synthesis. below, we describe the core components of both systems. the baseline system includes three modules: asr, llm, and tts. the asr transcribes user speech to text, which is encoded and compared against a news database for relevant article retrieval. the llm generates a response based on both the transcript and retrieved news snippets, and the tts outputs spoken responses in a default tone. we utilize retrieval augmented generation (rag) in our system. the core asr and rag-llm module are shared: the asr transcript is passed to a rag language model that selectively retrieves news to ground its replies, using dynamic incontext prompting for adaptability. our proposed system enhances the baseline with a sentiment analyzer that infers the emotional tone (neutral, happy, sad, angry, or surprised) from the llm's text response. the emotion tag is fed to prompttts, an emotional tts module that conditions speech synthesis on both text and emotion, producing expressive and empathetic responses. compared to the emotionally neutral baseline, our system delivers more human-like, engaging interactions through sentiment understanding and emotional prosody. ## experiments to evaluate the extent to which the proposed method can control proper speech emotion, we compared it with a baseline system using human subjective judgments. ## conclusion we presented an emotional sds that enhances empathetic interactions in task-oriented news conversations by combining a sentiment analyzer with prompttts for dynamic emotional speech generation. our system integrates sentiment-driven emotional control within a prompt-based architecture, improving emotion appropriateness and engagement without compromising task performance. its modular design enables easy adaptation to other domains, supporting broader development of emotionally aware conversational agents.