Flexible Thinking for Multimodal Emotional Support Conversation via Reinforcement Learning (2025)

Abstract: Emotional Support Conversation (ESC) systems aim to alleviate user distress. However, current Chain-of-Thought based ESC methods often employ rigid, text-only reasoning, limiting adaptability in dynamic, multimodal interactions and introducing reasoning noise that degrades support quality. To address this, we introduce “Flexible Thinking” for multimodal ESC, enabling models to adaptively select contextually relevant thinking aspects: Visual Scene, Emotion, Situation, and Response Strategy. We first construct training data by manually curating flexible thinking demonstrations on the MESC dataset, then using a Multimodal Large Language Model to synthesize these processes for the full training set. Then, we propose FIRES, a framework integrating Supervised Fine-Tuning (SFT) for initial learning with Reinforcement Learning for refinement. This two-stage approach helps FIRES transcend SFT’s generalization limits and, crucially, directly links thinking processes to response quality via tailored rewards, moving beyond imitating potentially imperfect synthetic data. Experiments on MESC and EMOTyDA datasets demonstrate FIRES’s effectiveness and generalizability in fostering higher-quality emotional support responses through adaptive reasoning.

## introduction emotional support and mental well-being are increasingly crucial in today's fast-paced society, where individuals commonly face pressures from both work and life. with the rapid advancements in artificial intelligence, providing emotional support through automated dialogue systems, known as emotional support conversation (esc) task, has emerged as a promising research topic of significant interest. the goal of esc is to alleviate users' emotional distress and help them cope with encountered challenges @xcite . it holds broad application prospects across various scenarios, such as daily interactions, social companionship, mental health support, and customer service, by improving user experiences through timely emotional support. early research on esc mostly focused on controlling response strategies @xcite @xcite or incorporating external knowledge @xcite @xcite to strengthen context understanding and improve response quality. in recent years, the ascent of chain-of-thought (cot) reasoning has inspired studies that simulate human-like step-by-step thinking before generating a response @xcite , aiming to enhance the interpretability and reliability. however, existing cot-based methods for esc mostly focus on text-only conversations and typically adopt fixed thinking steps @xcite @xcite . this design results in a rigid "onesize-fits-all" reasoning paradigm that struggles to adapt to the dynamic contexts of real-world conversations. in contrast, human supportive interactions are inherently multimodal (e.g., involving both textual and visual information), and human supporters do not mechanically deliberate on a fixed set of aspects; instead, they flexibly adjust their cognitive focus based on the specific context and the interlocutor's state. this flexibility is crucial for providing effective support. as shown in figure 1 , an empathetic response may require a deep analysis of the interlocutor's emotion cause and specific situation (example 1). when the available information in the limited conversational history is insufficient, quickly initiating further inquiry to gather more details is more effective than complex deep thinking (example 2). sometimes an appropriate response may critically hinge on the perception of events within the visual scene (example 3). a rigid thinking mode with fixed steps overlooks the actual utility of different reasoning aspects in specific contexts. when predefined thinking aspects mismatch current contextual requirements, such a mode not only fails to provide constructive guidance but may also introduce "reasoning noise", thereby undermining the quality of support. in this work, we introduce flexible thinking for the multimodal esc task, which aims to adaptively select and integrate those thinking aspects beneficial for generating high-quality emotional support responses based on the specific conversational context. specifically, we predefine four potential thinking aspects: perception of the current visual scene (visual scene), recognition of the interlocutor's emotions (emotion), analysis of the specific causes of emotions or the problems faced by the interlocutor (situation), and planning of the response strategy @xcite . to construct training data, we first manually annotated demonstrations of flexible thinking on the public mesc dataset @xcite . subsequently, we leverage the powerful multimodal large language model (mllm) to synthesize the flexible thinking processes for the entire training set based on these demonstrations. we propose fires (flexible thinking via reinforcement learning for multimodal emotional support conversation), a framework that synergizes supervised fine-tuning (sft) and reinforcement learning (rl). initially, sft on synthesized data teaches the model foundational thinking pat-terns. however, sft alone has limitations, as models may memorize flaws from the synthetic data (like inaccurate inferences or redundant steps) and struggle to generalize by imitating these potentially imperfect examples. to overcome this, employs the latest rl algorithm, group relative policy optimization (grpo), which is derived from recent work on reasoning models @xcite . this rl stage allows the model to explore diverse reasoning paths and learn from carefully designed reward function that directly links the thinking process to response quality, promoting effective reasoning beyond mere imitation of synthetic data. we conduct extensive experiments using both open-source and closed-source mllms on the mesc dataset. the results demonstrate the advantages of flexible thinking and the superiority of the proposed fires. further experiments on another multimodal dialogue dataset emotyda @xcite also validate the generalizability of our framework. our contributions are summarized as follows: (1) we introduce the concept of flexible thinking to the multimodal esc task, encouraging the model to, based on the specific conversational context, adaptively integrate aspects that facilitate effective response generation. (2) we propose the fires framework, which involves an initial cold start via sft and a subsequen rl optimization through grpo coupled with a customized reward function for esc, to stimulate the model towards effective flexible thinking and the generation of high-quality emotional support responses. (3) experimental results demonstrate the effectiveness and generalizability of our framework. ## conclusion in this paper, we introduce "flexible thinking" to address the limitations of rigid, fixed-step, and text-only reasoning. for the multimodal esc task, we propose a framework named fires to cultivate helpful flexible thinking and generate effective response via a two-stage process: an initial sft cold start on flexible thinking data synthesized by a mllm, followed by refinement using an rl algorithm with a meticulously designed reward function. extensive experiments on the mesc and emotyda datasets have demonstrated the effectiveness and generalizability of our framework. we believe that enabling models to think flexibly like humans, i.e., adaptively adjusting their thinking focus based on the evolving multimodal context, is a promising path toward developing more humanlike conversational ai capable of providing genuine and helpful emotional support.