bluuebunny/crossref_metadata_2025 huggingface dataset:
ProcDefense — A Game Framework for Procedural Player Skill Training
Authors: Thorne Brandon , Nelakkutti Hiru , Reinhart Joseph , Jhala Arnav
Published: October 2022

A challenge of game design is in providing affordances to players so that they can learn and improve their skills. Ad- vances in Procedural Content Generation (PCG) suggest this type of game content is a candidate for automatic creation. Some work in PCG has been successful in customizing game difficulty to achieve desired player experience; however, this often involves bringing the difficulty of the game to a level appropriate for the player’s current skills. Players desiring to improve their performance in a particular game may be will- ing to tolerate relatively higher levels of frustration and anx- iety than are targeted in experience-based approaches. As an initial step in this line of inquiry, we introduce ProcDefense, an action game with a modular difficulty control interface, as a platform for future inquiry into the effectiveness of differing PCG techniques for performance-training, dynamic difficulty adjustment.
Social Simulation for Social Justice
Authors: Dickinson Melanie , Wardrip-Fruin Noah , Mateas Michael
Published: October 2022

We argue that social simulation can help us understand social justice issues. In particular, modeling certain social dynamics within computational systems can be used to creatively explore and better understand the social and identity dynamics of oppression. Writing theories of oppression in code forces us to explicate everything, and question what we leave out or what we can’t account for. As an early step in this direction, we present an in-progress social simulation of group discussion in activist meetings, developed in the already-existing AI system, Ensemble. Through this minimal, highly constrained social arena, we can explore wide-reaching phenomena like privilege, intersectionality, and power dynamics in nonhierarchical groups, but in a way that’s grounded in concrete, person-to-person interactions. We propose that this kind of social simulation can aid in the process of unlearning hegemonic ways of being, and imagining liberatory alternatives.
A Specialized Corpus for Film Understanding
Authors: Winer David , Magliano Joseph , Clinton James , Osterby Aidan , Ackerman Thomas , Young R.
Published: October 2022

We have developed a cinematic annotation scheme and used that scheme to create a shot-by-shot annotated corpus of similar scenes to aid with research in film understanding. This paper introduces and describes the scheme and the created corpus, discusses some of its merits, and summarizes some basic findings.
Sketching a Generative Model of Intention Management for Characters in Stories: Adding Intention Management to a Belief-Driven Story Planning Algorithm
Authors: Young R. Michael
Published: October 2022

Previous work on story planning has shown success in the generation of plans that are both intention-coherent and demonstrate aspects of inter-character conflict. However, the initial models of intention and conflict have been limited, in that they lack methods to generate story plots wherecharacters drop sub-plans to achieve their goals in believably consistent and expressive ways and adopt new sub-plans in the face of plan failure. In current work, we have developed models of failed actions in stories that go hand in hand with erroneous belief models for character. Motivated by characterizations of rational agents' intentions as choice combined with commitment, we provide a framing of the plan generation process that is intended to show how characters form their own plans to achieve their own goals, act upon those plans until they feel that conditions no longer support their plans, and then re-plan in the face of adversity to achieve their goals. We show an example story plan that contains several types of character-based intention dynamics targeted by our approach.
Building on Word Animacy to Determine Coreference Chain Animacy in Cultural Narratives
Authors: Jahan Labiba , Chauhan Geeticka , Finlayson Mark
Published: October 2022

Animacy is the characteristic of being able to independently carry out actions in a story world (e.g., movement, communication). It is a necessary property of characters in stories, and so detecting animacy is an important step in automatic story understanding. Prior approaches to animacy detection have conceived of animacy as a word- or phrase-level property, without explicitly connecting it to characters. In this work we compute the animacy of referring expressions using a statistical approach incorporating features such as word embeddings on referring expression, noun, grammatical subject and semantic roles. We then compute the animacy of coreference chains via a majority vote of the animacy of the chain's constituent referring expressions. We also reimplement prior approaches to word-level animacy to compare performance. We demonstrate these results on a small set of folktales with gold-standard annotations for coreference structure and animacy (15 Russian folktales translated into English). Folktales present an interesting challenge because they often involve characters who are members of traditionally inanimate classes (e.g., stoves that walk, tree that talk). We achieve an F1 measure 0.90 for the referring expression animacy model, and 0.86 for the coreference chain model. We discuss several ways in which we anticipate these results may be improved in future work.
Dynamic and Accelerated Partial Order Planning for Interactive Narratives
Authors: Zhang Xun , Inampudi Bhuvana , Badler Norman , Kapadia Mubbasir
Published: October 2022

This paper explores new narrative generation paradigms for open world problems. We propose a speed-up variant of partial planner–accelerated partial order planner, that can automatically generate narratives for large plan spaces. To incorporate real-time free-form user interaction, a dynamic partial planning technique has been introduced to self-repair the narratives. We also propose a scalable and robust framework to craft open world narratives with minimal effort. Our approach enables content creators to craft complex open world narratives without explicitly authoring user interaction arcs. We tested our framework by developing multiple narratives with free-form interactions. Those narratives were used to test the robustness of the proposed planners.
Analyzing Expressionist Grammars by Reduction to Symbolic Visibly Pushdown Automata
Authors: Osborn Joseph , Ryan James , Mateas Michael
Published: October 2022

We extend the Expressionist project, and thereby the re-emerging area of grammar-based text generation, by applying a technique from software verification to a critical search problem related to content generation from grammars. In Expressionist, authors attach tags (corresponding to pertinent meanings) to nonterminal symbols in a context-free grammar, which enables the targeted generation of content that expresses requested meanings (i.e., has the requested tags). While previous work has demonstrated methods for requesting content with a single required tag, requests for multiple tags yields a search task over domains that may realistically span quintillions or more elements. In this paper, we reduce Expressionist grammars to symbolic visibly pushdown automata, which allows us to locate in massive search spaces generable outputs that satisfy moderately complex criteria related to tags. While the satisficing of more complex tag criteria is still not feasible using this technique, we forecast a number of opportunities for future directions.
Towards Intelligent Narrative-Based Interfaces for Sensemaking
Authors: Niehaus James , Reilly Scott , Weyhrauch Peter , Tittle James , Young R.
Published: October 2022

Professionals such as military analysts, medical providers, and business administrators have a significant need to understand and make sense of an increasing quantity of data. We propose intelligent narrative-based interfaces as a means of organizing, presenting, and interacting with this data to im-prove user comprehension. This paper identifies critical questions and potential research paths in intelligent narrative-based interfaces for sensemaking. We propose a general architecture of intelligent narrative-based interfaces and identify central research questions in this area. We believe this new area of research has significant potential to use narrative technologies to impact a wide range of real world problems.
Experience Management with Beliefs, Desires, and Intentions for Virtual Agents
Authors: Farrell Rachelyn
Published: October 2022

Intelligent interactive narrative systems often use an experience manager to govern the behavior of non-player charactersin a way that guides the story towards its author’s agenda, which may be for entertainment, education, training, or other purposes. For such systems, a central challenge is creating believable virtual characters. The Belief Desire Intention framework is often cited as a goal for researchers in this field; for characters to seem realistic, a human audience should attribute beliefs, desires, and intentions to them. Much of my prior work has focused on belief; my goal for the future is to finish the work on belief, and to implement a new model of desire and intention that explicitly reasons about characters’ commitment to certain plans of action.
Structure Editors and Autonomous Agents
Authors: Card Alexander
Published: October 2022

Rational agents are becoming prevalent in many domains, from data analysis to entertainment and games. The increased prevalence of agents has evolved new tools and techniques to work with and design new agents. One such technique is system simulation. Systems simulation is a technique an author can use to imitate tasks, processes, or systems, and in particular, agents. Systems simulation has a variety of uses, ranging from simulating ecological systems to entertainment, such as interactive narratives and digital games. However, many system simulators use specialized programming languages and require prior programming experience. This causes a disconnect between individuals with limited programming experience who wish to use the simulation tools, and the software itself. New users may find the specialized languages daunting, and the initial learning process too intense for the anticipated reward. This research strives to bridge the gap between system simulation tools and users with little to no programming experience. Future work includes a corpus of narrative and autonomous agent creation tools designed for users with little to no programming experience.
VERTIGØ: Visualisation of Rolling Horizon Evolutionary Algorithms in GVGAI
Authors: Gaina Raluca , Lucas Simon , Pérez-Liébana Diego
Published: October 2022

This report presents a tool developed for the analysis and visualisation of Rolling Horizon Evolutionary Algorithms, featuring a GUI which allows integration within the General Video Game AI Framework. Users are able to easily customize the parameters of the agent between runs and observe an in-depth analysis of its performance through various visual information extracted from gameplay data, live while playing the game. This visualisation aims to inform a deeper analysis into algorithm behaviour, in an attempt to justify why they make the decisions they do and improve their performance based on this knowledge.
Expanding Expressive Range: Evaluation Methodologies for Procedural Content Generation
Authors: Summerville Adam
Published: October 2022

Procedural Content Generation (PCG) has been a part of video games for the majority of their existence and have been an area of active research over the past decade. How- ever, despite the interest in PCG there is no commonly ac- cepted methodology for assessing and analyzing a generator. Furthermore, the recent trend towards machine learned PCG techniques commonly state the goal of learning the design within the original content, but there has been little assess- ment of whether these techniques actually achieve this goal. This paper presents a number of techniques for the assess- ment and analysis of PCG systems, allowing practitioners and researchers better insight into the strengths and weaknesses of these systems, allowing for better comparison of systems, and reducing the reliance on ad-hoc, cherry-picking-prone tech- niques.
Predicting Generated Story Quality with Quantitative Measures
Authors: Purdy Christopher , Wang Xinyu , He Larry , Riedl Mark
Published: October 2022

The ability of digital storytelling agents to evaluate their output is important for ensuring high-quality human-agent interactions. However, evaluating stories remains an open problem. Past evaluative techniques are either model-specific--- which measure features of the model but do not evaluate the generated stories ---or require direct human feedback, which is resource-intensive. We introduce a number of story features that correlate with human judgments of stories and present algorithms that can measure these features. We find this approach results in a proxy for human-subject studies for researchers evaluating story generation systems.
Improbotics: Exploring the Imitation Game Using Machine Intelligence in Improvised Theatre
Authors: Mathewson Kory , Mirowski Piotr
Published: October 2022

Theatrical improvisation (impro or improv) is a demanding form of live, collaborative performance. Improv is a humorous and playful artform built on an open-ended narrative structure which simultaneously celebrates effort and failure. It is thus an ideal test bed for the development and deployment of interactive artificial intelligence (AI)-based conversational agents, or artificial improvisors. This case study introduces an improv show experiment featuring human actors and artificial improvisors. We have previously developed a deep-learning-based artificial improvisor, trained on movie subtitles, that can generate plausible, context-based, lines of dialogue suitable for theatre. In this work, we have employed it to control what a subset of human actors say during an improv performance. We also give human-generated lines to a different subset of performers. All lines are provided to actors with headphones and all performers are wearing headphones. This paper describes a Turing test, or imitation game, taking place in a theatre, with both the audience members and the performers left to guess who is a human and who is a machine. In order to test scientific hypotheses about the perception of humans versus machines we collect anonymous feedback from volunteer performers and audience members. Our results suggest that rehearsal increases proficiency and possibility to control events in the performance. That said, consistency with real world experience is limited by the interface and the mechanisms used to perform the show. We also show that human-generated lines are shorter, more positive, and have less difficult words with more grammar and spelling mistakes than the artificial improvisor generated lines.
Exploring Gameplay With AI Agents
Authors: De Mesentier Silva Fernando , Borovikov Igor , Kolen John , Aghdaie Navid , Zaman Kazi
Published: October 2022

The process of play testing a game is subjective, expensive and incomplete. In this paper, we present a play-testing approach that explores the game space with automated agents and collects data to answer questions posed by the designers. Rather than have agents interacting with an actual game client, this approach recreates the bare bone mechanics of the game as a separate system. Our agent is able to play in minutes what would take testers days of organic gameplay. The analysis of thousands of game simulations exposed imbalances in game actions, identified inconsequential rewards and evaluated the effectiveness of optional strategic choices. Our test case game, The Sims Mobile, was recently released and the findings shown here influenced design changes that resulted in improved player experience.
A Monte Carlo Approach to Skill-Based Automated Playtesting
Authors: Horn Britton , Miller Josh , Smith Gillian , Cooper Seth
Published: October 2022

In order to create well-crafted learning progressions, designers guide players as they present game skills and give ample time for the player to master those skills. However, analyzing the quality of learning progressions is challenging, especially during the design phase, as content is ever-changing. This research presents the application of Stratabots — automated player simulations based on models of players with varying sets of skills — to the human computation game Foldit. Stratabot performance analysis coupled with player data reveals a relatively smooth learning progression within tutorial levels, yet still shows evidence for improvement. Leveraging existing general gameplaying algorithms such as Monte Carlo Evaluation can reduce the development time of this approach to automated playtesting without losing predicitive power of the player model.
Modeling Player Experience with the N-Tuple Bandit Evolutionary Algorithm
Authors: Kunanusont Kamolwan , Lucas Simon , Pérez-Liébana Diego
Published: October 2022

Automatic game design is an increasingly popular area of research that consists of devising systems that create content or complete games autonomously. The interest in such systems is two-fold: games can be highly stochastic environments that allow presenting this task as a complex optimization problem and automatic play-testing, becoming benchmarks to advance the state of the art on AI methods. In this paper, we propose a general approach that employs the N-Tuple Bandit Evolutionary Algorithm (NTBEA) to tune parameters of three different games of the General Video Game AI (GVGAI) framework. The objective is to adjust the game experience of the players so the distribution of score events through the game approximates certain pre-defined target curves. We report satisfactory results for different target score trends and games, paving the path for future research in the area of automatically tuning player experience.
Talin: A Framework for Dynamic Tutorials Based on the Skill Atoms Theory
Authors: Aytemiz Batu , Karth Isaac , Harder Jesse , Smith Adam , Whitehead Jim
Published: October 2022

Most tutorials in video games do not consider the skill level of the player when deciding what information to present. This makes many tutorials either tedious for experienced players or not informative enough for players who are new to the given genre. With Talin, implemented as an asset in the Unity game engine, we make it possible to create a mastery model of an individual player's skill levels by operationalizing Dan Cook's skill atom theory. We propose that using this mastery model opens up a new design space when it comes to designing tutorials. We show an example tutorial implementation with Talin assembled using only graphical components provided by our framework, without the need of writing any code. The dynamic tutorial implementation results in the player receiving information only when they need it, whenever they need it. While the novice player is given all the information they need to learn the system, the expert player is not bogged down by tooltip pop-ups regarding mechanics they have already mastered.
Agent Modeling as Auxiliary Task for Deep Reinforcement Learning
Authors: Hernandez-Leal Pablo , Kartal Bilal , Taylor Matthew E.
Published: October 2022

In this paper we explore how actor-critic methods in deep reinforcement learning, in particular Asynchronous Advantage Actor-Critic (A3C), can be extended with agent modeling. Inspired by recent works on representation learning and multiagent deep reinforcement learning, we propose two architectures to perform agent modeling: the first one based on parameter sharing, and the second one based on agent policy features. Both architectures aim to learn other agents’ policies as auxiliary tasks, besides the standard actor (policy) and critic (values). We performed experiments in both cooperative and competitive domains. The former is a problem of coordinated multiagent object transportation and the latter is a two-player mini version of the Pommerman game. Our results show that the proposed architectures stabilize learning and outperform the standard A3C architecture when learning a best response in terms of expected rewards.
Is Your Game Generator Working? Evaluating Gemini, an Intentional Generator
Authors: Osborn Joseph C. , Dickinson Melanie , Anderson Barrett , Summerville Adam , Denner Jill , Torres David , Wardrip-Fruin Noah , Mateas Michael
Published: October 2022

Determining whether a game generator is working properly is challenging, since it entails conducting potentially many evaluations of generated games and synthesizing these into a net evaluation of the system. The problem is compounded when the generator has a human-centered goal: for example, that the generated games should be interpreted as having certain mechanics or as being about particular ideas. In this paper, we examine the Gemini game generator and develop an evaluation instrument that tests the interpretability of its generated games’ mechanics and higher-order proceduralist arguments. In the process we build empirical evidence for the claim that some amount of non-systems-based framing is required in order for arguments made by procedural rhetorics to be sensible to players. The tools we have assembled for this evaluation can be applied to game generators more broadly; game generators should be allowed to invent games which go beyond merely formally “good” or subjectively “fun.”
Macro Action Selection with Deep Reinforcement Learning in StarCraft
Authors: Xu Sijia , Kuang Hongyu , Zhi Zhuang , Hu Renjie , Liu Yang , Sun Huyang
Published: October 2022

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also widely accepted as a challenging testbed for AI research because of its enormous state space, partially observed information, multi-agent collaboration, and so on. With the help of annual AIIDE and CIG competitions, a growing number of SC bots are proposed and continuously improved. However, a large gap remains between the top-level bot and the professional human player. One vital reason is that current SC bots mainly rely on predefined rules to select macro actions during their games. These rules are not scalable and efficient enough to cope with the enormous yet partially observed state space in the game. In this paper, we propose a deep reinforcement learning (DRL) framework to improve the selection of macro actions. Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as LastOrder. Our evaluation, based on training against all bots from the AIIDE 2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning rate, outperforming 26 bots in total 28 entrants.
Augmenting Character Path Planning with Layered Social Influences
Authors: Chen Zeyuan , McCoy Josh
Published: October 2022

Creating behavior for believable characters is one of the main challenges within video games and is a key component in providing immersive experiences. As games become more complex, developers turn to AI techniques to generate believable character behaviors to enhance a player’s gameplay experience. Traditionally, path planning allows characters to navigate game worlds in a responsive way but has run-time efficiency and path distance as its primary goals. However, this tradition can be broken to make path planning another channel for generating believable characters. This paper describes a system of augmenting path planning with layered influences that can capture different social problems and nuances of character interactions within games. Additionally, we explore the social importance of characters moving through game worlds via a system that allows authors, game makers, and characters to use path planning as an expressive channel.
Action Guidance with MCTS for Deep Reinforcement Learning
Authors: Kartal Bilal , Hernandez-Leal Pablo , Taylor Matthew E.
Published: October 2022

Deep reinforcement learning has achieved great successes in recent years, however, one main challenge is the sample inefficiency. In this paper, we focus on how to use action guidance by means of a non-expert demonstrator to improve sample efficiency in a domain with sparse, delayed, and possibly deceptive rewards: the recently-proposed multi-agent benchmark of Pommerman. We propose a new framework where even a non-expert simulated demonstrator, e.g., planning algorithms such as Monte Carlo tree search with a small number rollouts, can be integrated within asynchronous distributed deep reinforcement learning methods. Compared to a vanilla deep RL algorithm, our proposed methods both learn faster and converge to better policies on a two-player mini version of the Pommerman game.
Cooperation and Codenames: Understanding Natural Language Processing via Codenames
Authors: Kim Andrew , Ruzmaykin Maxim , Truong Aaron , Summerville Adam
Published: October 2022

Codenames – a board game by Vlaada Chvátil – is a game that requires deep, multi-modal language understanding. One player, the codemaster, gives a clue to another set of players, the guessers, and the guessers must determine which of 25 possible words on the board correspond to the clue. The nature of the game requires understanding language in a multi-modal manner – e.g., the clue ‘cold’ could refer to temperature or disease. The recently proposed Codenames AI Competition seeks to advance natural language processing, by using Codenames as a testbed for multi-modal language understanding. In this work, we evaluate a number of different natural language processing techniques (ranging from neural approaches to classical knowledge-base methods) in the context of the Codenames AI framework, attempting to determine how different approaches perform. The agents are evaluated when working with identical agents, as well as evaluated with all other approaches – i.e., when they have no knowledge about their partner.
Beyond Winning and Losing: Modeling Human Motivations and Behaviors with Vector-Valued Inverse Reinforcement Learning
Authors: Wang Baoxiang , Sun Tongfang , Zheng Xianjun Sam
Published: October 2022

In recent years, reinforcement learning (RL) methods have been applied to model gameplay with great success, achieving super-human performance in various environments, such as Atari, Go and Poker. However, these studies mostly focus on winning the game and have largely ignored the rich and complex human motivations, which are essential for understanding humans’ diverse behavior. In this paper, we present a multi-motivation behavior model which investigates the multifaceted human motivations and learns the underlying value structure of the agents. Our approach extends inverse RL to vectored-valued rewards with Pareto optimality which significantly weakens the inverse RL assumption. Our model therefore incorporates a wider range of behavior that commonly appears in real-world environments. For practical assessment, our algorithm is tested on World of Warcraft datasets and demonstrates the improvement over existing methods.
Standardizing the Evaluation of Digital Managers for Better Interactive Experiences
Authors: Mori Giulio
Published: October 2022

As Artificial Intelligence (AI) technologies become increasingly integrated with society (e.g., in the form of digital assistants, games, or self-driving cars), it becomes increasingly important to understand how they can ensure that people’s experiences meet certain desirable criteria. In Experience Management, researchers and professionals seek to create automated systems called “experience managers”, which work to improve people’s experiences in designer-specified ways. Unfortunately, the evaluation and comparison of experience managers is currently impaired by the lack of both a common platform to evaluate them and a common language to describe experience management tasks. Experience in other fields such as General Game Playing and AI Planning has shown that having a common platform can stimulate and accelerate research progress. The goal of this project is thus to develop a platform for evaluating and comparing arbitrary experience managers on a wide variety of tasks, to accelerate the essential progress of Experience Management research. We plan to do this by studying existing experience managers, developing a common platform that can support them, and promoting the platform to the research community as a tool for their research. Throughout our work, we will draw lessons from the field of General Game Playing, hoping to reuse some of its existing infrastructure and stimulate new exchanges of ideas between the two fields of research.
How Actors Can Animate Game Characters: Integrating Performance Theory in the Emotion Model of a Game Character
Authors: Schiffer Sheldon
Published: October 2022

Despite the development of sophisticated emotion models, game character facial animation is still often completed with laborious hand-controlled key framing, only marginally assisted by automation. Behavior trees and animation state machines are used mostly to manage animation transitions of physical business, like walking or lifting objects. Attempts at automated facial animation rely on discrete facial iconic emotion poses, thus resulting in mechanical “acting”. The techniques of acting instructor and theorist Sanford Meisner reveal a process of role development whose character model resembles components of Appraisal Theory. The similarity conjures an experiment to discover if an “emotion engine” and workflow method can model the emotions of an autonomous animated character using actor-centric techniques. Success would allow an animation director to autonomously animate a character’s face with the subtlety of gradient expressions. Using a head-shot video stream of one of two actors performing a structured Meisner-esque improvisation as the primary data source, this research demonstrates the viability of an actor-centric workflow to create an autonomous facial animation system.
Playable Experiences at the 15th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE’19)
Authors: Liu Rook , Christopher Claire , Martens Chris , Ontañón Santiago , Mirowski Piotr , Mathewson Kory W. , Winters Thomas , Farrugia Shaun
Published: October 2022

This paper describes the accepted entries to the seventh Playable Experiences track to be held at the 15th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE’19). The Playable Experiences track showcases innovative complete works that are informed, inspired, or otherwise enabled by artificial intelligence.
Narrative Planning for Belief and Intention Recognition
Authors: Farrell Rachelyn , Ware Stephen
Published: October 2022

Planning algorithms generate sequences of actions that achieve a goal, but they can also be used in reverse: to infer the goals that led to a sequence of actions. Traditional plan-based goal recognition assumes agents are rational and the environment is fully observable. Recent narrative planning models represent agents as believable rather than perfectly rational, meaning their actions need to be justified by their goals, but they may act in ways that are not optimal, and they may possess incorrect beliefs about the environment. In this work we propose a technique for inferring the goals and beliefs of agents in this context, where rationality and omniscience are not assumed. We present two evaluations that investigate the effectiveness of this approach. The first uses partial observation sequences and shows how this impacts the algorithm’s accuracy. The second uses human data and compares the algorithm’s inferences to those made by humans.
PCGRL: Procedural Content Generation via Reinforcement Learning
Authors: Khalifa Ahmed , Bontrager Philip , Earle Sam , Togelius Julian
Published: October 2022

We investigate how reinforcement learning can be used to train level-designing agents. This represents a new approach to procedural content generation in games, where level design is framed as a game, and the content generator itself is learned. By seeing the design problem as a sequential task, we can use reinforcement learning to learn how to take the next action so that the expected final level quality is maximized. This approach can be used when few or no examples exist to train from, and the trained generator is very fast. We investigate three different ways of transforming two-dimensional level design problems into Markov decision processes, and apply these to three game environments.
Video Game Level Repair via Mixed Integer Linear Programming
Authors: Zhang Hejia , Fontaine Matthew , Hoover Amy , Togelius Julian , Dilkina Bistra , Nikolaidis Stefanos
Published: October 2022

Recent advancements in procedural content generation via machine learning enable the generation of video-game levels that are aesthetically similar to human-authored examples. However, the generated levels are often unplayable without additional editing. We propose a “generate-then-repair” framework for automatic generation of playable levels adhering to specific styles. The framework constructs levels using a generative adversarial network (GAN) trained with human-authored examples and repairs them using a mixed-integer linear program (MIP) with playability constraints. A key component of the framework is computing minimum cost edits between the GAN generated level and the solution of the MIP solver, which we cast as a minimum cost network flow problem. Results show that the proposed framework generates a diverse range of playable levels, that capture the spatial relationships between objects exhibited in the human-authored levels.
Word Autobots: Using Transformers for Word Association in the Game Codenames
Authors: Jaramillo Catalina , Charity Megan , Canaan Rodrigo , Togelius Julian
Published: October 2022

Winning the social game Codenames involves combining cooperative and language understanding capabilities. We developed six cooperative bots designed to play the Codemaster and Guesser roles in the Codenames AI Competition and tested them using the provided framework and a round-robin tournament set. The bots are based on term frequency - inverse document frequency (TF-IDF), Naive-Bayes and GPT-2 Transformer word embedding. Additionally, Transformer-based bots were assessed and compared with the concatenation of word2vec and GloVe baseline bot developed by Codenames AI Competition creators. Results from this Transformer implementation rivals the concatenated bot in terms of win rates and guess precision and outperforms it in terms of minimum and average turns taken to win the game and training data load time. Additionally, in an initial evaluation performed with 10 human players, the Transformer agent performed slightly better than the baseline as Codemaster, but worse as a Guesser.
PAIndemic: A Planning Agent for Pandemic
Authors: Sauma-Chacón Pablo , Eger Markus
Published: October 2022

Cooperative games are an important challenge for AI research. One example of this genre of games is the board game Pandemic, characterized by the challenges it presents, including its considerable search space and hidden information, especially when the cooperators are not able to communicate explicitly. The game pushes players to prioritize between short term and long term objectives, which forces them to not only plan their own individual actions but also to cooperate with the other players in order to win the game. In this paper we present a planning agent which uses state-space planning with a domain-specific heuristic, combined with a Monte Carlo sampling approach to predict possible outcomes in the face of hidden information. We performed several experiments with our agent, including a comparison with a baseline version that does not use planning. Our experiments showed that our agent is able to win about a third of the games played in a realistic game setup.
Combinatorial Q-Learning for Dou Di Zhu
Authors: You Yang , Li Liangwei , Guo Baisong , Wang Weiming , Lu Cewu
Published: October 2022

Deep reinforcement learning (DRL) has gained a lot of attention in recent years, and has been proven to be able to play Atari games and Go at or above human levels. However, those games are assumed to have a small fixed number of actions and could be trained with a simple CNN network. In this paper, we study a special class of Asian popular card games called Dou Di Zhu, in which two adversarial groups of agents must consider numerous card combinations at each time step, leading to huge number of actions. We propose a novel method to handle combinatorial actions, which we call combinatorial Q-learning (CQL). We employ a two-stage network to reduce action space and also leverage order-invariant max-pooling operations to extract relationships between primitive actions. Results show that our method prevails over other baseline learning algorithms like naive Q-learning and A3C. We develop an easy-to-use card game environments and train all agents adversarially from sractch, with only knowledge of game rules and verify that our agents are comparative to humans. Our code to reproduce all reported results is available on github.com/qq456cvb/doudizhu-C.
A Sketch-Based Tool for Authoring and Analyzing Emergent Narratives
Authors: Kybartas Ben , Verbrugge Clark , Lessard Jonathan
Published: October 2022

We present a real-time tool for authoring and analyzing emergent narratives (EN) built using a possible worlds EN framework. The tool works by reducing the emergent narrative into a discrete tension space (TS), based on possible conflicts between characters, and actions treated as movements in the space. The shape and structure of the TS enables an analysis of higher-level narrative and character properties. Likewise, TS can be constructed through a sketch-based interface, that enables the creation of context-free emergent narrative systems.
Towards Designing Out Helplessness: AI Interventions for Videogame Learnability
Authors: Aytemiz Batu
Published: October 2022

I research how we can utilize artificial intelligence (AI) techniques to make videogames more effective teachers of their own internal complexities. I focus on exploring how we can design more graceful tutorials by finding inspiration from the learning sciences and inclusive design domains. Towards this end I study theories of failure in videogames, build tools that emphasize best learning practices and develop AI systems that improve videogame learnability.
A Case Study in Learning in Metagames: Super Smash Bros. Melee
Authors: Codsi Julien , Vetta Adrian
Published: October 2022

Imagine agents repeatedly playing a bimatrix game against opponents drawn from a population of assorted skill levels.
This paper studies how agents strategize in such a metagame and the population distributions that result.
Specifically, we investigate how an agent should adjust its strategy
as it also learns to play the game, that is, as the agent
improves its skills (from novice to expert) with repeated exposure to the game.
To perform this task, we introduce a dynamic game-theoretic model of learning in metagames. We use it to explain the learning dynamics and character selection exhibited in data from the game Super Smash Bros. Melee. Indeed, the primary motivation behind this work is the application of game-theoretic methods in video game balancing.
Plug-and-Blend: A Framework for Plug-and-Play Controllable Story Generation with Sketches
Authors: Lin Zhiyu , Riedl Mark O.
Published: October 2022

Large pre-trained neural language models (LM) have very powerful text generation capabilities.
However, in practice, they are hard to control for creative purposes.
We describe a Plug-and-Play controllable language generation framework, Plug-and-Blend, that allows a human user to input multiple control codes (topics).
In the context of automated story generation, this allows a human user loose or fine-grained control of the topics and transitions between them that will appear in the generated story, and can even allow for overlapping, blended topics.
Automated evaluations show our framework, working with different generative LMs,
controls the generation towards given continuous-weighted control codes while keeping the generated sentences fluent, demonstrating strong blending capability.
A human participant evaluation shows that the generated stories are observably transitioning between two topics.
Efficient 2D Sound Propagation in Video Games
Authors: Davari Nima , Verbrugge Clark
Published: October 2022

Sound is well known for improving immersion in video games. In many games sound also serves as an important game mechanism; in stealth games, for example, sounds triggered by player actions may attract enemy agents, and sounds created by enemy agents give useful information on where to target or avoid combat. Modelling sound, however, is computationally complex, and thus sound propagation is often restricted to highly localized contexts, or based on trivial propagation models such as a fixed radius that ignores the impact of obstacles. In this work, we describe an efficient approach to propagating sound in a real-time 2D game level. We use a heuristic ray-based approach, adapted to include diffraction, reflection, and transmission. Our design allows for dynamic geometries, avoids sound discontinuities, and can be tuned in various ways to trade-off accuracy for computation cost.
Generating Tailored Advice in Video Games through Play-Style Identification and Player Modelling
Authors: Ingram Branden
Published: October 2022

Recent advances in fields such as reinforcement learning have enabled the development of systems that are able to achieve super-human performance on a number of domains, specifically in complex games such as Go and StarCraft. Based on these successes, it is reasonable to ask if these learned behaviours could be utilised to improve the performance of humans on the same tasks. However, the types of models used in these systems are typically not easily understandable, and can not be directly used to improve the performance of a human. My research looks to address these difficulties by developing a system that can provide advice tailored to a player's style in a video game setting. This system would be particularly useful for improving tutorial systems that quickly become redundant lacking any personalization. I have already developed an unsupervised approach to identifying different play-styles present in a video game. I look to use this knowledge to train agents which can demonstrate optimal behaviour for each style. It is hoped that this information can be utilised to generate useful advice based on the differences between the current performance of a player and the corresponding expert agent in their style.
A Tool for Generating Monster Silhouettes with a Word-Conditioned Variational Autoencoder
Authors: Gonzalez Adrian , Guzdial Matthew , Ramos Felix
Published: October 2022

A character’s appearance is crucial to communicating game mechanics to the audience. Creating a game character’s design is a time-consuming task and requires design knowledge, skills, and experience. Research on how an AI system might be able to support this design process is an underexplored area. In this work we present a prototype of a variational autoencoder-based creativity support tool that modifies game character silhouettes by using words to describe the design’s desired properties.
A Demonstration of KiaiTime: A Mixed-Initiative PCGML Rhythm Game Editor
Authors: Halina Emily , Guzdial Matthew
Published: October 2022

Chart creation for rhythm action games is a time consuming task that requires specialized design knowledge. While chart generation systems have been explored in the past, there are currently no co-creative chart authoring systems. In this paper, we present KiaiTime, a mixed-initiative, co-creative PCGML editor for the rhythm game Taiko no Tatsujin. KiaiTime allows the user to interface with an AI partner that acts as a source of guidance and inspiration in the chart creation process.
A Demonstration of Blabrecs, an AI-Based Wordgame
Authors: Kreminski Max , Karth Isaac
Published: October 2022

Blabrecs is an AI-based modification to the popular wordgame Scrabble. In Blabrecs, English dictionary words may not be played; instead, players may only play nonsense words that are approved by a classifier trained on a list of English dictionary words. Gameplay therefore revolves around inventing plausibly English-sounding nonsense words and learning how to fool the classifier. In this paper, we briefly introduce our design goals for Blabrecs; describe the process by which Blabrecs was designed; and present two distinct implementations of the game's AI judge.
Enhancing Multimodal Goal Recognition in Open-World Games with Natural Language Player Reflections
Authors: Gupta Anisha , Carpenter Dan , Min Wookhee , Rowe Jonathan , Azevedo Roger , Lester James
Published: October 2022

Open-world games promote engagement by offering players a high degree of autonomy to explore expansive game worlds. Player goal recognition has been widely explored for modeling player behavior in open-world games by dynamically recognizing players’ goals using observations of in-game actions and locations. In educational open-world games, in-game reflection tools can help students reflect on their learning and plan their strategies for future gameplay. Data generated from students’ written reflections can serve as a source of evidence for modeling player goals. We present a multimodal goal recognition approach that leverages players’ written reflections along with game trace log features to predict player goals during gameplay. Results show that both the highest predictive performance and best early prediction performance are achieved by deep learning-based, multimodal goal recognition models that utilize both written reflection and gameplay features as input. These models outperform unimodal deep learning models as well as a random forest baseline. Multimodal goal recognition using natural language reflection data has significant potential to enhance goal recognition model performance, as well as player modeling more generally, to support the creation of engaging and adaptive open-world digital games.
Minimizing Coordination in Multi-Agent Path Finding with Dynamic Execution
Authors: Wagner Aidan , Veerapaneni Rishi , Likhachev Maxim
Published: October 2022

Multi-agent Path Finding (MAPF) is an important problem in large games with many dynamic agents that need to follow space-time trajectories without inter-agent collisions. Modern MAPF solvers plan assuming that agents directly follow the space-time trajectories at known constant speeds without delays or speedups, resulting in rigid plans which need to be replanned if there are changes during execution. Instead we would like agents to be able to follow their computed paths with dynamic velocities while requiring minimal coordination with others to prevent collisions and deadlocks. One way to address this problem is to first produce collision free space-time paths and then compute a coordination controller that prevents collisions and deadlock during dynamic execution. This two step process prevents fully minimizing coordination as the initially planned space-time paths do not reason about coordination and can be arbitrarily bad. We introduce a novel paradigm and show how planning in space-coordination level, rather than space-time, allows us to simultaneously plan paths and a coordination controller. Our method, Space-Level Conflict-Based Search (SL-CBS), builds on the Conflict-Based Search framework and allows us to reason explicitly about coordination, producing paths as well as a coordination controller with bounded suboptimal minimal coordination. We show experimentally that this results in a 20-50% reduction in coordination compared to the closest state of the art solver.