Instruction-Driven Game Engine: A Poker Case Study (2024)

Abstract: The Instruction-Driven Game Engine (IDGE) project aims to democratize game development by enabling a large language model (LLM) to follow free-form game descriptions and generate game-play processes. The IDGE allows users to create games simply by natural language instructions, which significantly lowers the barrier for game development. We approach the learning process for IDGEs as a Next State Prediction task, wherein the model autoregressively predicts the game states given player actions. The computation of game states must be precise; otherwise, slight errors could corrupt the game-play experience. This is challenging because of the gap between stability and diversity. To address this, we train the IDGE in a curriculum manner that progressively increases its exposure to complex scenarios.Our initial progress lies in developing an IDGE for Poker, which not only supports a wide range of poker variants but also allows for highly individualized new poker games through natural language inputs. This work lays the groundwork for future advancements in transforming how games are created and played.

## introduction game developers dedicate creativity to offer immersive experiences to game players. players immerse themselves in games and offer valuable feedback to developers. this makes a symbiotic relationship between creators and customers. however, as depicted in the comic from figure 1 , there are disconnections between them, due to diverse preferences of players across age, gender, and cultural backgrounds. despite the fact that many today's games allow for customization of basic characters and appearances, it is an impossible task for developers to craft every aspect of the game to suit the need of every player. our study seeks to reconcile such a divide. game engines, as the heart of game development, are conventionally driven by programming languages. this technical barrier often deters enthusiasts from realizing their game development dreams. in response, we propose a novel concept: instruction-driven game engine (idge), a game engine enabling anyone to fashion a game through natural language instructions and generating the resultant game-play process. distinct from recent advancements in video-based games @xcite , our focus in this paper is on the text-based game states. we leverage unity to render these states to visual display. idge is a neural engine, meaning it is built upon neural networks, specifically large language models (llms) @xcite @xcite . it is designed to follow a game script, a detailed instruction that blueprints the game, e.g. settings, rules, elements, and drive the progression of game-play as interacting with players. idges frame the operation of engines as a next state prediction task, which autoregressively predicts the next game state based on the user-specified game script, previous game state, and current player action. training an idge faces the dual challenges of stability and diversity. the former seeks to provide a stable and precise game-play throughout lengthy contexts, while the latter seeks to follow diverse preferences across the large player base. unfortunately, we empirically see an ironic twist: the model trained directly from naive game logs is neither stable nor diverse. therefore, we employ a standard-to-diverse curriculum learning methodology, which gradually introduces complexity into the training process, incrementally enhancing the model's diversity while preserving its stability. developer: in our new game, players will control... player: why protagonist is always a man? developer: yes, maybe we should... developer: in dual protagonists, you can always switch between a man and a woman. player: why always men or women? developer: emmm... developer: please tell your thought to our engine. 1 2 3 4 5 player: design the protagonist as a martian with black skin. player: give the protagonist the superpower of instantaneous moving. ## instruction-driven game engine in this section, we introduce dialogue-style llms as the setup for idges. we then formulate the learning problem as next state prediction. ## data for idge our training data is sourced from two methods. first, we develop a poker simulator and obtain the training data from its game logs. the simulator supports ten representative poker games: texas @xmath0: {'players': ['p1', 'p2', 'p3']}, 'chips': { 'p1': {'bet': 0, 'remain': 1000}, 'p2': {'bet': 0, 'remain': 1000}, 'p3': {'bet': 0, 'remain': 1000} }, 'deck': ['h1', 'd1', 'c1', 's1', 'h2', . hold'em, omaha, omaha hl, short-deck hold'em, 2-to-7 triple draw, a-to-5 triple draw, 2-to-7 single draw, badugi, badeucey, and badacey. additionally, it allows for further configuration of several common poker elements, e.g. type of suits, numbers. by adjusting these elements, one can derive virtually infinite variations beyond aforementioned ten poker games. moreover, we realize that if the game logs are sampled completely in uniform, the occurrence of some rare states, such as some superior card combinations, would be extremely low. the resultant engine trained on such data may fall short in low-frequency situations, even though the dataset is large. therefore, we balance the data by up/down-sampling the game logs to ensure that all possible situations occur similarly. after obtaining game logs, we transform each log into a training sample as in figure 2 for nsp and dsp. each sample is made up of three parts: the game script z, player input x t , and game states s t . if we were to draw an analogy with chatgpt, they respectively play the roles of the system, user, and assistant. the second part of the data is generated by gpt3.5. based on the state prediction data from the simulator, we prompt gpt3.5 to augment the game scripts and generate the corresponding new game states. this process is manually done by skilled prompting, which incorporates scenarios beyond typical poker games, expanding the diversity of training data as a result. ♠ game script to describe the poker game in natural language, we design a prototype game script. the top part of figure 2 illustrates the game script for a texas hold'em variant. we can see that it defines a series of game elements: the number of players in the game, minimum and maximum bet limits, suits and values of single cards, battle strategies, and the game flow. these elements correspond to the configuration of the poker simulator. particularly, the game flow refers to the procedures this game will go through in order, e.g. bet, deal. ♠ game state and player input for the game state and player input, we adopt a dict format as shown on the left side of figure 2 . for instance, "deck" is followed by the remaining cards in the deck, "hole" and "community" is followed by the hole cards of players and the public cards, while "message" is followed by the message sent from the source a to target b. on the right side of figure 2 , we show the diagram of the poker game associated to the left-side game state. in player input n , player 2 chooses to raise the bet. given state n -1, the engine outputs state n , where the chips of player 2 are updated and player 1 is informed to bet since player 3 has folded. to ensure the independence assumption that each state s t solely depends on state s t-1 , we incorporate the game flow as a summary module in the game state. it specifically caches all past game procedures in order. the engine can thus be navigated to step into the next procedure correctly, regardless of the amount of game-play history. data statistics ## curriculum learning straightforwardly, we could utilize the data generated in § 3 to fine-tune a base model by maximizing eq. 2/3 and obtain the idge. however, the resultant idge may struggle with stability and diversity: neither can it accurately predict the next game state nor comprehend the user-specified game script in natural language. therefore, we devise a progressive curriculum learning process @xcite , to incrementally enhance the idge's diversity while preserving stability. warmup: training on core set in § 2, we utilize a set of core functions to facilitate the process of state prediction. though the model can be exposed to all core functions via fine-tuning, we observe that it struggles to call the core functions properly. this phenomenon is much more severe in unseen contexts. we attribute this to the cold start problem that the model merely memorizes the names of the core functions during training, without knowing their underlying implementation. to this end, we introduce a pre-learning phase to warmup the model. we develop an instruction tuning dataset of 1k samples derived from the core set, where each core function is translated to a natural language instruction and the model is trained to implement the function in a way of instruction following. this phase offers a profound comprehension of the model's usage of core functions. ## experimental results in this section, we evaluate the idge in two scenarios. the former is automatically generated by our simulator, which can be considered as a test set that has the same distribution as the training set. the latter resembles the real-world situations, where proficient poker players are directly enlisted as annotators to create new game scripts. subsequently, the test data is obtained by playing the games online by themselves with the idge. ## conclusion this paper introduces the instruction-driven game engine (idge), offering game enthusiasts a brand new game development and game-play experience. the idge understands the player-specified game rules and simulates the entire game-play process. we formulate the learning of idges as next state prediction and leverage a curriculum learning approach to enhance stability and diversity. experiments demonstrate our poker idge can accurately complete the majority of user-defined games.