M ind C raft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks (2021)

Abstract: An ideal integration of autonomous agents in a human world implies that they are able to collaborate on human terms. In particular, theory of mind plays an important role in maintaining common ground during human collaboration and communication. To enable theory of mind modeling in situated interactions, we introduce a fine-grained dataset of collaborative tasks performed by pairs of human subjects in the 3D virtual blocks world of Minecraft. It provides information that captures partners’ beliefs of the world and of each other as an interaction unfolds, bringing abundant opportunities to study human collaborative behaviors in situated language communication. As a first step towards our goal of developing embodied AI agents able to infer belief states of collaborative partners in situ, we build and present results on computational models for several theory of mind tasks.

## introduction creating embodied, situated agents able to move in, communicate naturally about, and collaborate on human terms in the physical world has been a persisting goal in artificial intelligence @xcite . during communication in such a setting, agents not only need to ground entities in language to that of the physical world; efficient and accurate human-agent collaboration further requires agents to reason about the progress of the task at hand and to plan and execute a series of collaborative steps, whilst maintaining common ground @xcite with collaboration partners, in order to achieve a certain goal. despite recent advances, we are still far away from fully enabling these desired agent behaviors. one key challenge is in an agent's ability to establish and maintain common ground in tandem with human partners, especially in a setting where beliefs about the world and of each other may change on the fly @xcite . it is important to understand how changes in a dynamic, physical world affect agents' beliefs of each other-i.e., theory of mind @xcite -and how such beliefs influence teamwork and communication in collaborative tasks. as a first step to address this question, this paper explores theory of mind modeling @xcite @xcite in situated language communication @xcite for collaborative tasks within the 3d virtual blocks world of minecraft. through a novel experimental setup, we collect a situated dialogue dataset that demonstrates how collaborative partners with a set of asymmetric knowledge and skills are able to collaborate to achieve joint goals, and how, in particular, their beliefs of each other evolve and converge over time. based on this dataset, we further build several baseline computational models to explicitly predict key elements of a collaboration partner's mental state from the viewpoint of an agent as a task unfolds. our empirical results demonstrate that while language is certainly important in this inference, the shared physical environment and the perceived activities play a greater role in shaping a partner's understanding of each other in order to come to a common ground. the contributions of this work are threefold. first, we introduce mindcraft, a task in which pairs of users collaboratively work to create novel materials by combining blocks in the 3d virtual world of minecraft, with the ultimate objective of creating a final, goal material. unlike prior work in situated collaborative tasks @xcite @xcite , a key focus of our work is to facilitate theory of mind modeling-the ability to attribute mental states, both of one's own and that of others-an important but not yet wellstudied topic in situated collaborative interactions. within designed collaborative tasks, we have users record their beliefs about the state of the game, and of each other, at periodic intervals. our data captures an evolution of the states of mind of our participants that are true representations of their beliefs-not simply proxies for the true sequence of events in a collaborative session. this explicit modeling of theory of mind sheds light on how partners strive to align their mental models in order to achieve common ground during collaboration. second, departing from previous leader-follower setups (where one partner explicitly leads and gives instructions to the other, the follower, who tries to execute said instructions) @xcite , we focus on a setting where partners each have asymmetric knowledge @xcite and skill-sets towards completing a joint goal. in order to effectively complete the given tasks, partners need to negotiate their own plans of action by taking into account what they currently know and don't know about their partner, and of their common understanding of the task at hand. our novel, more relaxed setup provides support for greater diversity in modes of collaboration that are more representative of that in the real world. third, we introduce a set of baseline computational models to infer fellow player mental states in situ, as a collaborative agent would, and highlight some further challenges present in moving towards building fully realistic agents able to reason about human mental states in situated environments. our platform, data, and models are made available ‡ and will facilitate future work on physical agents that can effectively collaborate with humans through situated dialogue. situated and task-oriented natural language interactions @xcite have been studied in a variety of environments, ‡ https://github.com/sled-group/ mindcraft including in custom 2d worlds @xcite @xcite , in the physical world with human-robot interactions @xcite @xcite , and in various 3d virtual worlds @xcite . most closely, our environment builds upon recent work by @xcite and @xcite , whereby computational models of user dialogue prediction and user next-action prediction are investigated in the setting of a collaborative dialogue task within the 3d virtual blocks world of minecraft. however, to our knowledge, none of these previous works explicitly model theory of mind for dialogue agents. theory of mind as a subject, especially in computation @xcite , has gained increased attention in areas including agent-agent reinforcement learning @xcite , dialogue systems @xcite , human-computer interaction @xcite , agent-agent collaborative dialogue @xcite , and explainable ai @xcite . worthy of note is the type of mental state recording we employ: specifically, we ask players to record their own mental states during interaction. unlike prior work that has largely utilized external annotators for post-hoc mental state attribution , we expand on @xcite and @xcite by specifically bringing user self-reported mental states from that of only the linguistic domain to multimodal situated dialogue. specifically, the novelty in our work exists in studying and bringing explicit theory of mind modeling to 3d situated collaborative interactions. ## experimental system and data collection we consider a scenario whereby two agents, situated in the same environment and able to perform actions simultaneously, collaborate to complete a shared goal. here, unlike traditional leader-follower setups, both agents have asymmetric information on the steps needed to complete the target task. in addition, in certain iterations, agents have asymmetric skill-sets as well: certain steps may only be completed with specific skills, and an agent may not be able to complete the target task by themselves, even with complete knowledge. agents are provided a text channel to communicate in natural language, where they are able to share knowledge and negotiate on the actions to be per-figure 1 : diagram of a sample interaction in mindcraft. two players are tasked to complete a common goal within the game environment of minecraft; players communicate using in-game chat, are provided partial views of the plan needed to create the goal material, and are periodically asked paired questions to probe into their mental states. additionally, we record first-person viewpoint videos of the two players' points of view (pov) as well as a third-person pov from the shared game environment. ## findings and observations we further perform an analysis of our dataset to gain an understanding of collaborative behaviors between players both in their reasoning and in their alignment of mental models. ## computational models for inferring belief states based on our dataset, a variety of computational problems can be formulated, developed, and evaluated. in this section, we focus on one key problem-predicting player belief states of the task and of a collaborative partner in situ. as a first step, we implement a straightforward model that, from a player's perspective, predicts the state of the task as well as the mental states of a partner at any given time based on historical observations of a rich discourse of dialogue and perceived actions in the shared environment. figure 4 shows the overall architecture of our model. our dataset is comprised of two time-seriesbased modalities: (1) a video stream coming from either player's first-person pov, and (2) dialogue exchanges. we implement a forward sequence-tosequence model, such that inferences at any given time are only able to process inputs that have occurred before it. ## evaluation of belief state inference we randomly partition our dataset into 60%/20%/20% training, validation, and testing splits with the condition that all three partitions have a similar distribution of game lengths. to achieve testing in situ, we replace one of the two players with our model. at every point where a question is prompted, our model is used to provide an answer about the other player's belief state through inference, using the self-reported belief state of the other player as ground truth for evaluation. we present the multi-class average f1 score weighted by the number of instances in each class (accounting for class imbalances) in our results. we perform our experiments by varying ## conclusion and future work in a real-world collaborative scenario with physical agents, humans and agents will inevitably have disparities in their abilities, knowledge, and understanding of the shared world. this work specifically stimulates these disparities in a virtual environment, introducing a new dataset and experimental framework that supports in-depth studies of theory of mind modeling for situated dialogue in collaborative tasks. through a novel implementation of self-reported belief states during collaborative interactions, our dataset keeps track of partners' beliefs about the task at hand and of each other step-by-step and captures how their states of mind evolve-and, indeed, how their common ground evolves-as communication and interaction unfold. to our knowledge, this is the first dataset in the context of situated dialogue that provides this fine-grained information for mental modeling. our initial analysis of this dataset generates several interesting findings that will inform the development of computational models for various problems-for instance, in tracking mental models and managing dialogue behaviors in collaborative agents. our baseline results demonstrate the importance of interaction discourse and visual experience in a shared environment on predicting mutual belief states of the task at hand, and of a collaborative partner, to ensure common ground. while we have built baseline computational models to better help in understanding human collaborative behaviors and several theory of mind tasks, we hope our work further facilitates improvements in areas like agent planning and decision-making, computational reasoning, multimodal dialogue generation, and to move towards fully autonomous agents that are able to engage with humans in collaborative activities, on human terms, both effectively and efficiently in a human world. exempt. a total of 29 subjects took part in the data collection; no personally identifiable information was stored throughout our experimental sittings, and participants were provided with anonymous minecraft accounts to access the game servers such that they did not use their own. we do not additionally control for any ethnicity or cultural aspects aside from the condition that the participant is to be an english speaker and has some experience with minecraft.