I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons (2023)

Abstract: We propose a novel task, G4C, to study teacher-student natural language interactions in a goal-driven and grounded environment. Dungeons and Dragons (D&D), a role-playing game, provides an ideal setting to investigate such interactions. Here, the Dungeon Master (DM), i.e., the teacher, guides the actions of several players—students, each with their own personas and abilities—to achieve shared goals grounded in a fantasy world. Our approach is to decompose and model these interactions into (1) the DM’s intent to guide players toward a given goal; (2) the DM’s guidance utterance to the players expressing this intent; and (3) a theory-of-mind (ToM) model that anticipates the players’ reaction to the guidance one turn into the future. We develop a novel reinforcement learning (RL) method for training a DM that generates guidance for players by rewarding utterances where the intent matches the ToM-anticipated player actions. Human and automated evaluations show that a DM trained to explicitly model intents and incorporate ToM of the players using RL generates better-quality guidance that is 3x more likely to fulfill the DM’s intent than a vanilla natural language generation (NLG) approach.

## introduction humans communicate with a goal in mind and use language to reach the goal by interacting with their communication partners grounded in a shared environment @xcite @xcite . to make sure the goal is reached, we often anticipate how the partners will respond in advance to steer the conversations in the desired direction. this ability to reason about the mental states of conversation partners -i.e., theory-of-mind (tom; @xcite -is key to smooth and efficient communication @xcite . most existing dialogue agents, while able to produce human-like responses, often do not model ## g4c and g-dragon here we discuss how we construct the environment for the proposed g4c task using a dataset of dialogues from dungeons and dragons (d&d) called g-dragon. we start with formulating the g4c task, then introduce the d&d data, and finally present our procedure of constructing the environment using large-scale data. ## theory-of-mind inspired guidance generation in grounded environments this section introduces our exploration of model designs to train a teacher model that can guide the student to perform certain actions by speaking in a grounded environment. we are specifically interested in the research question "does incorporating intent (3.1) and theory-of-mind (3.2) help models generate better guidance?" ## evaluating g4c here we propose multifaceted evaluation protocols to measure the quality of the dm/teacher model for g4c. we introduce three criteria, fluency, groundedness, and goal-fulfillment, to evaluate model outputs. we design automatic metrics and human evaluation protocols for each criterion, and analyze how well the proposed metrics correlate with human judgments in 5.2. we refer to outputs satisfying all three criteria as star dm. ## experimental results we aim to answer three research questions through our experiments: 1) do idm-provided labels help train models that generate better guidance? 2) does explicitly incorporating intents result in better models? 3) does theory-of-mind modeling help models become better communicators? ## conclusion we propose g4c to study goal-driven and grounded language interactions focusing on generating guidance from the teacher to lead students to perform certain actions. we use d&d as our test bed and construct large-scale data g-dragon by using idm to provide quality labels. we train models to generate guidance by modeling intents and theory-of-mind. results show a promising sign that incorporating explicit intents and tom modeling makes better communication agents.