Towards Socially Intelligent Agents with Mental State Transition and Human Value (2022)

Abstract: Building a socially intelligent agent involves many challenges. One of which is to track the agent’s mental state transition and teach the agent to make decisions guided by its value like a human. Towards this end, we propose to incorporate mental state simulation and value modeling into dialogue agents. First, we build a hybrid mental state parser that extracts information from both the dialogue and event observations and maintains a graphical representation of the agent’s mind; Meanwhile, the transformer-based value model learns human preferences from the human value dataset, ValueNet. Empirical results show that the proposed model attains state-of-the-art performance on the dialogue/action/emotion prediction task in the fantasy text-adventure game dataset, LIGHT. We also show example cases to demonstrate: (i) how the proposed mental state parser can assist the agent’s decision by grounding on the context like locations and objects, and (ii) how the value model can help the agent make decisions based on its personal priorities.

## introduction recently, there has been remarkable progress in language modeling with large-scale pretrained models @xcite @xcite . such models are used to build either general chatbots @xcite or task-oriented dialogue systems @xcite @xcite . while most of these systems have been able to generate fluent sentences, there are two major challenges towards building socially intelligent agents. first, considering dialogues as a "meeting of minds" @xcite or achieving some alignment of the interlocutors' mental models @xcite , few existing works are explicitly * equal contribution. the work was done prior to liang joining amazon alexa. tracking the mental state transition of agents @xcite . endowing current dialogue systems with such capability would allow the agent to condition its utterance on the context, simulate the effect of its actions, and further help understand the extended meaning, implicature, and irony expressed by the user @xcite . second, it remains under-explored to teach agents to make a rational decision guided by its value. from a social and cultural perspective, humans tend to have a common preference described by the utility function related to individual values, common sense, and social awareness. for the example in figure 1 , someone who values personal security prefers staying at home rather than going outside at night. our work aims to alleviate the aforementioned problems, based on embodied cognitive linguistics (ecl) @xcite and established value theories in sociology @xcite . the ecl states that natural language is inherently executable, driven by mental simulation and metaphoric inference @xcite , and learned through embodied interaction @xcite . following its tenents, we present a hybrid mental state parser that converts dialogue and event observations into a graphical representation of the agents' mind. initialized with the location and object description, the interpretable representation is updated through the interaction history to track the evolving process of an agent's belief about surroundings and other agents. in the field of intercultural research, schwartz (1992); @xcite identify basic individual values that are recognized across cultures. inspired by the theory, we propose to incorporate a value model that learns social common preferences from the human value knowledge base, val-uenet @xcite . we perform experiments on a large-scale text-based embodied environment light @xcite . empirical results show that the model with our mental state emulator and value function achieves the highest performance that aligns with human annotation among existing transformer-based models. moreover, case studies further demonstrate that the mental state provides extra context information, while the value model helps agents make value-driven decisions. our contributions are two-fold. first, we propose to rethink the design of current dialogue systems and suggest a new paradigm from the perspective of cognitive science and contemporary sociology. second, we present a new framework for building socially intelligent agents by incorporating mental state simulation and human value modeling into dialogue generation and decision making. our methodology can be generalized to a wide range of interactive social situations in dialogue systems @xcite , virtual reality @xcite , and human-robot interactions @xcite . ## problem formulation we will first briefly introduce the text-adventure environment light, followed by the mental state modeling and value utility formulation. light @xcite is a large-scale crowd-sourced fantasy text-adventure platform for studying grounded dialogues. figure 4 a ⃝ shows a typical local environment setting, including location description, objects (and their affordances), characters, and their personas. agents can talk to other agents in free-form text, take actions defined by templates, or express certain emotions (figure 4 b ⃝). given the environmental setting and observation history, our task is to predict the agent's utterance/action/emotion for the next turn. to achieve this goal in a socially intelligent manner, we model the agent's mental state transition and incorporate human values. the mind model is proposed to depict the agent's belief about the underlying states of the text world. meanwhile, a utility function of human values is designed to describe human preferences in common social situations. we experiment on the text-adventure game for simplicity, but the proposed architecture supports richer environments. ## algorithms the overall architecture of our proposed framework is illustrated in figure wearing, or wielding. the free-form conversations, actions, and emotions are logged during the communication as the observation history (figure 4 b ⃝). to begin with, a mental state parser will parse the setting descriptions into graph representation and initialize the agent's mental state (steps 1 ## experiments we conduct experiments on the light dataset and compare our model with state-of-the-art methods based on two variants of bert models. an ablation study is carried out to justify our model design, and a case study is performed to demonstrate how the proposed framework could help the agent ground upon the environment details and make value-driven decisions. ## conclusion this paper proposes to build a socially intelligent agent by incorporating mind simulation and human values. we explore using a hybrid parser to track agents' mental state transition. the value model pretrained on valuenet brings social preference to help the agent make decisions. the model is proved to have a better performance than the stateof-the-art models on light. in the future, we have a plan to build a dataset to study the implicature in conversation and model deeper levels in the theory of mind based on the proposed representation.