Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence (2022)

Abstract: AI researchers have posited Dungeons and Dragons (D&D) as a challenge problem to test systems on various language-related capabilities. In this paper, we frame D&D specifically as a dialogue system challenge, where the tasks are to both generate the next conversational turn in the game and predict the state of the game given the dialogue history. We create a gameplay dataset consisting of nearly 900 games, with a total of 7,000 players, 800,000 dialogue turns, 500,000 dice rolls, and 58 million words. We automatically annotate the data with partial state information about the game play. We train a large language model (LM) to generate the next game turn, conditioning it on different information. The LM can respond as a particular character or as the player who runs the game—i.e., the Dungeon Master (DM). It is trained to produce dialogue that is either in-character (roleplaying in the fictional world) or out-of-character (discussing rules or strategy). We perform a human evaluation to determine what factors make the generated output plausible and interesting. We further perform an automatic evaluation to determine how well the model can predict the game state given the history and examine how well tracking the game state improves its ability to produce plausible conversational output.

## introduction artificial intelligence has a long and rich history of using games as challenge problems that lead to advances in the field. in many cases, ai game-playing systems have gone on to rival human champions of the game. dungeons and dragons has been identified as an appropriate challenge for the next stage of artificial intelligence @xcite @xcite . @xcite proposed open-ended creative games like d&d as the next challenge for ai after the human-level successes of ai at chess and * *denotes equal contribution go, which are zero-sum, deterministic, sequential two-player games with perfect information. @xcite understood the importance of narrative in natural language processing (nlp) and generation (nlg). in particular, they saw how cooperative story generation between humans already exists in these games and can be used for automated generation. @xcite outlined some of the specific challenges d&d presents to the nlp community; such as a state of the game world distributed across the dungeon master (dm) and other players or dealing with the intrinsic rewards players get from taking certain actions that would not necessarily provide them with points in the game. d&d involves multiple players who roleplay characters in a fantasy setting, guided by a dungeon master who sets obstacles and adventures and plays as monsters. in roleplaying games like dungeons and dragons, the gameplay happens through language rather than moves on a game board, making it an interesting domain for nlp research. to have an ai successfully play d&d , it would require abilities like is it possible to design an ai system that is capable of playing a game of d&d either as a character in the game or as the dungeon master using current ai technology? we argue that now is the perfect time for this challenge, since large scale neural language models like gpt have shown impressive generation results @xcite , and since incorporating neural lms into a game setting both exercises their strengths and exposes their weaknesses. in this paper, we introduce a new dataset of "actual play" game transcripts. each turn is labeled with game state variables like character information and whether the conversational turn was incharacter or out-of-character. our data is a novel, large scale, real-world conversational dataset. it is unique in that the dialog turns are generated entirely through player collaboration and written interaction in a multi-player game. we propose our dataset as a challenge for dialogue systems for the following reasons: ## tasks we trained a large language model (llm) to perform two tasks: next utterance prediction and game state tracking. ## dataset for this paper, we have created a novel dataset for our dialogue-oriented test of ai's ability to play dungeons & dragons. we scraped play-by-post data from a web forum foot_0 where people play by taking turns posting on the forum to describe their move. figure 1 shows an example of part of the gameplay from the play-by-post forums from d&d beyond. d&d beyond provides a mechanism in its forum to roll dice using a "roll" tag. their dice roller allows players to conduct the rolls that are used for d&d ability checks and in combat. ## models for our large language model, we use a 64b parameter version of google's lamda language model @xcite , which was trained on conversations. lamda is similar to other transformerbased pre-trained language models like gpt-3. as with other pre-trained language models @xcite , lamda can be finetuned to different tasks. the two tasks that we finetune lamda to perform are game state tracking and response generation. in both cases, the llm can be thought of as a function that maps inputs onto an output. for instance, game state tracking is a language understanding task where the function takes in inputs like f (current utterance, previous state, history) → new state, and response generation is a language generation task where f (current state, history) → next utterance. the llm functions are trained via the fine tuning process. in our experiments we try a variety of different inputs to our llm functions to see how they enable better learning of the tasks. we train our llms on the conversation history (which is typical in dialog modeling) and we also augment the conversations by conditioning other explicit signals. these conditioning signals can be thought of as sophisticated "control features", inspired by the ctrl language model @xcite . during training, the model learns a relationship between the control features and appropriate responses. in turn, during inference, one can explicitly influence dimensions of the conversation -enabling more compelling dialogue -by setting the values of control features. these control features can be set dynamically, without necessitating finetuning or additional post-processing. ## manual evaluation to evaluate the quality of our models for the task of next utterance prediction in d&d , we perform a human evaluation. we recruited professional raters to perform a manual evaluation. they read a version of the content that was provided to the models -the seven turns of conversational history plus a list of players and the names/classes of the characters that they played. then they were shown several model outputs for the context (or the "gold", which was the actual next turn in the game), the annotators asked to rate each output along the three dimensions, following the evaluation procedure used for the meena lm @xcite ): ## game state tracking model we conducted an experiment to evaluate whether a llm could be finetuned to perform game state tracking for d&d using our heuristically annotated game state features. we trained a new model llm-dnd-gst (game state tracking). it is a llm-dialog that has been finetuned on our play-by-post d&d gameplay dataset. as input, it takes all previous dialog turns and their state variables, plus the text of the current turn, and then it outputs the corresponding state variables for the current turn. we analyzed the accuracy of the llm-dnd-gst model its ability to do slot-filling for each of the individual game states, and compared its performance to a simple baseline that always output the the majority class. the results are shown in ## related work previous work has examined ai to play text adventures games @xcite @xcite . these games are simpler than d&d because they have a limited vocabulary and more straightforward game states. creating text adventure games @xcite is more challenging than playing them, and is similar to the world-building job of the dm in d&d . there has also been work on persona/character generation in stories @xcite , and within d&d itself @xcite . others @xcite have realized that npcs are lacking in their abilities to speak and act in text games. findings of the automated story generation community are relevant for d&d ai systems. neural language models have become increasingly more popular for story generation @xcite @xcite . we have also started to see storytelling with transformers @xcite @xcite . transformer-based storytelling systems have even been introduced to the general public thanks to the popularity of ai dungeon @xcite . although neural networks possess a lot of power in terms of what text they generate, they are still limited in their ability to produce longer spans of coherent text. many @xcite @xcite @xcite have improved the coherence of neural storytellers by splitting the generation into two steps: ideation of the story plot, followed by the realization of sentences. this controllable story generation is the focus of a lot of current work in neural automated story generation. due to the conversational nature of d&d , we decided to use a dialog-based system. deep neural networks have been used for dialog agents for a while @xcite , with a shift toward using transformers in recent years @xcite . like in automated story generation and other neural text generation tasks, we are also seeing controllability being an important factor being integrated into systems. this includes using deep reinforcement learning techniques to guide the dialog toward a goal @xcite or controlling for style @xcite . in this paper, we use lamda, a transformerbased open-domain dialogue system that builds on the meena model @xcite . the original meena model was an end-to-end model trained on public conversations found on social media. controllable text generation with transformers has been seen before with ctrl @xcite , a language model that is conditioned on a given "control code" in addition to the textual history. this work takes a similar approach. we integrate contextual information such as character descriptions, actions, and in-and out-of-character classifications. we have finetuned our lamda models on data crawled from d&d beyond. this data contains both in-character and out-of-character dialog and can be used in conjunction with @xcite 's dataset from critical role (a d&d podcast), louis and sutton (2018)'s dataset from roleplayerguild.com (a d&d forum), urbanek et al. (2019)'s crowdsourced light dataset, and/or akoury et al. (2020)'s storium dataset for human+ai collaborative story generation. ## discussion and conclusions we have demonstrated that training on d&d data results in much higher quality outputs than a vanilla dialogue system (as expected), that controlling the model to generate in-character responses results in substantially more interesting output, and that conditioning on game state information qualitatively results in responses that are appropriate to the character class. our preliminary experiments with using the large language models to perform game state tracking show low performance even after finetuning, suggesting that other models may be required for an ai to play d&d track the full state of the game. although our models are unable to play d&d fully autonomously by acting as the dungeon master, they could act as an aid for novice dms. since our models can generated evocative, in-character text that is appropriate for the context and the game state, dms could use it as inspiration as they narrate the adventure to the other players. here is some model output to inspire your next adventure: you get a much closer look than the other two... the sarcophagi have the inscriptions of some sort of magic, probably to keep the dead inside, but you can not read them to save your life. what will you do next? download our dataset foot_4 to start your new adventure!