Self-Emotion Blended Dialogue Generation in Social Simulation Agents (2024)

Abstract: When engaging in conversations, dialogue agents in a virtual simulation environment may exhibit their own emotional states that are unrelated to the immediate conversational context, a phenomenon known as self-emotion. This study explores how such self-emotion affects the agents’ behaviors in dialogue strategies and decision-making within a large language model (LLM)-driven simulation framework. In a dialogue strategy prediction experiment, we analyze the dialogue strategy choices employed by agents both with and without self-emotion, comparing them to those of humans. The results show that incorporating self-emotion helps agents exhibit more human-like dialogue strategies. In an independent experiment comparing the performance of models fine-tuned on GPT-4 generated dialogue datasets, we demonstrate that self-emotion can lead to better overall naturalness and humanness. Finally, in a virtual simulation environment where agents have free discussions, we show that self-emotion of agents can significantly influence the decision-making process of the agents, leading to approximately a 50% change in decisions.

## introduction in an artificial social environment such as an openworld video game, it is crucial to have nonplayer characters reflect believable conversational ability @xcite and express human-level emotions @xcite . during conversations, a speaker's expressed emotion typically comprises a blend of emotions stemming from the conversational context, denoted as context-emotion, and those arising from life events tangential to the ongoing conversation, denoted as self-emotion @xcite . consider a scenario where speaker a informs speaker b that she has passed the bar exam (see figure 1 ). the context-emotion recognized in this scenario could be one of joy or impressed. however, the emotion expressed by speaker b significantly varies when influenced by different selfemotions triggered by other events. for example, b might exhibit more intense happiness and an "excited" emotion if b is also experiencing a positive event (e.g., a promotion). conversely, a negative event (e.g., failing an exam) can decrease the happiness associated with the context-emotion, leading b to express a "disappointed" emotion. despite its critical impact on dialogue behavior, self-emotion is often overlooked in the design of recent dialogue models. in this work, we take the approach of representing self-emotion as events derived from simulated background world of speakers using large language models (llms) and explore the extent to which self-emotion influences conversational behaviors of an agent. to achieve this, we construct a virtual agent framework and observe the dialogue behaviors of the agents under various self-emotional states. specifically, the agents in our framework are simulated to experience a series of events over a period of time, with the transitions in their self-emotional states caused by these events being tracked. at random points in time, the agents engage in conversations with each other, with their self-emotional states aligned with their "experienced events." in this manner, we analyze how the agents exhibit different dialogue behaviors, such as employing various strategies and setting different goals. in an experiment comparing conversations generated by llm-driven agents, with and without consideration of self-emotion, the results show that agents are able to generate more human-like dialogue strategies incorporating self-emotion. furthermore, results from a model comparison experiment show that conversations incorporating selfemotion are evaluated as more natural, empathetic, and human-like for both gpt-4 and a small-scale flan-t5 model fine-tuned on a gpt-4 generated dataset. finally, in a simulated group discussion experiment where agents discuss five different topics, we observe that the self-emotion of the agents significantly influences the decision-making process, resulting in approximately a 55% change in decisions. our contributions in this work include: ## related work self-emotion self-emotion, also referred to as "internal emotion," plays a significant role in daily interactions. research on group discussions indicates that self-emotion in individuals can affect the quality of decisions @xcite , team performance @xcite , and the decision-making process itself @xcite . furthermore, other studies suggest that the self-emotion of one member can influence others through a mechanism known as mood contagion @xcite . individual self-emotion has also been shown to impact dialogue strategies @xcite . in their research, @xcite demonstrate that negative self-emotion encourages more accommodative thinking. additionally, other studies suggest that effective self-emotion management contributes to the development of leadership skills @xcite . emotion-aware dialogue generation existing emotion-aware dialogue models typically begin by recognizing an emotion label from the conversation history and then proceed with conditional text generation based on that recognized emotion label. the most common emotion representation used is discrete emotion categories, such as the ekman basic emotions @xcite . subsequent studies have further refined emotion labels to include more than 30 categories @xcite @xcite . some works also represent emotions using different styles, such as intensity @xcite , causalities in history @xcite , and potential emotion transitions @xcite . however, the limitation of this approach is that it assumes the emotional state of speakers depends solely on the ongoing conversation discourse. our work differs from these approaches in that we consider self-emotion, which exists outside the conversation context. in this sense, our approach is similar to response generation based on user profiles @xcite @xcite . ## self-emotion agents framework we build a framework 1 in which agents' selfemotional states are influenced by a series of events generated by llms according to their profiles. agents in this framework are prompted to manage their own self-emotion, goals, actions, and profiles. ## self-emotion in strategy selection the purpose of this experiment is to explore whether incorporating self-emotion leads to more human-like dialogue strategies. in this experiment, we have agents simulate speakers in the empa-theticdialogues (ed) dataset and select the best strategies from a predefined strategy pool in two situations: with and without self-emotion. we then compare the strategies provided by the models to those made by human experts and evaluate the accuracy. ## self-emotion in dialogue generation in this experiment, we explore whether incorporating self-emotion in a dialogue model leads to better performance of the generated conversations using gpt-4. additionally, considering the challenges associated with deploying large language models ## self-emotion in group discussion self-emotion can influence group discussions @xcite . in this experiment, agents in the simulated world within our framework are prompted to engage in group discussions incorporating self-emotion across five topics related to teamwork. the purpose of this experiment is to explore how the self-emotion of agents may affect the decision-making process during a discussion. ## conclusion this work studies the role that self-emotion, speaker's emotion status caused by out-of-context events plays in the process of generating emotional responses. via a human evaluation, we show that models considering self-emotion are able to generate more natural conversations with more humanlike strategies. in an experiment of group discussion simulation, we also show that agent with selfemotion can have significant influence on the decision making process. the results of the experiments demonstrate the importance of considering self-emotion when building embodied agents and dialogue models that can smoothly participate in human social activities.