GAVEL: Generating Games via Evolution and Language Models
Authors: Graham Todd
Alexander George Padula
Matthew Stephenson
Eric Piette
Dennis J. N. J. Soemers
Julian Togelius
Venue: NeurIPS 2024 poster (2024)
Keywords: games
llms
language models
evolution
quality diversity
pcg
Primary area: generative_models
TL;DR: We use evolutionary computation and large language models to generate new and interesting board games

Abstract: Automatically generating novel and interesting games is a complex task. Challenges include representing game rules in a computationally workable form, searching through the large space of potential games under most such representations, and accurately evaluating the originality and quality of previously unseen games. Prior work in automated game generation has largely focused on relatively restricted rule representations and relied on domain-specific heuristics. In this work, we explore the generation of novel games in the comparatively expansive Ludii game description language, which encodes the rules of over 1000 board games in a variety of styles and modes of play. We draw inspiration from recent advances in large language models and evolutionary computation in order to train a model that intelligently mutates and recombines games and mechanics expressed as code. We demonstrate both quantitatively and qualitatively that our approach is capable of generating new and interesting games, including in regions of the potential rules space not covered by existing games in the Ludii dataset.

GAVEL: Generating Games Via Evolution and Language Models
Graham Todd
Alexander G. Padula ETH Zurich Zurich, Switzerland apadula
Matthew Stephenson Flinders University Adelaide, Australia
New York University Tandon Brooklyn, New York, USA gdrtodd
matthew. stephenson
Éric Piette UCLouvain
Dennis J.N.J. Soemers Maastricht University Maastricht, the Netherlands dennis soemers'
Julian Togelius
New York University Tandon Brooklyn, New York, USA
Louvain-la-Neuve Belgium
julian.togelive
@nyu. edu, @ethz.ck
@flinders .wwiddrrseedu.cc. au, @uclouvain.bo "emasstrichrunivesstty badcowaah...../maaatttttttttteee .nl,
Abstract
Automatically generating novel and interesting games is complex task. Challenges include representing game rules in a computationally workable form, searching through the large space of potential games under most such representations, and accurately evaluating the originality and quality of previously unseen games. Prior work in automated game generation has largely focused on relatively restricted rule representations and relied on domain-specifie heuristics. In this work, we explore the generation of novel games in the comparatively expansive Ludi i game description language, which encodes the rules of over 1000 board games variety of styles and modes of play. We draw inspiration from recent advances in large language models and evolutionary computation order to train a model that intelligently mutates and recombines games and mechanics expressed code. We demonstrate both quantitatively and qualitatively that our approach is capable of generating
and interesting games, including in regions of the potential rules space not covered by existing games in the Ludi dataset. sample of the generated games are available to play online through the Ludii portal.
Introduction
Games have long been used as test bed for algorithms and approaches in artificial intelligence, with advances in game-plan playiig ability often serving as some of the most recognizable achievements in the 55 65). While automated systems have repeatedly demonstrated ability to match or surpass humans as game players, they continue to lag significantly in their capacity to generate the kinds of games that are worth playing. The ability to construct novel and interesting games an impressive cognitive challenge, and success would have implications both cultural (with the
field J3 40
Play the generated games at: https:://udie wtt.a/agagaaeeeaeeeaaea details. 1.. botalls.ppppcccoor--aaaaa. httpa//eiaoiiaoooaaoooooooooooor and htttps///uuch games/
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Dataset
random expression from game taken mutoled using fllin-the-miide goneration
with
detaset constructed from games in LDG
Code LLM
balanced parenthatioal expression extracted
PCA Dinemiion Fileess
The ressiting
1. game selected from the archive
game is adtied to the
archive based on its
Phefic
ftness and the Luak
concepts
model langvagge
Taeger
PCA Dimanaion 1
the expression
Code LLM
Figure I: GAVEL overview. Left: a dataset of games in the Ludii game description language is used train code large language model using the fill-in-the-middte objective on parenthetical expressions. Right: the trained code language model can then be used as the mutation operator for evolutionary quality-diversity optimization with the MAP-Elites algorithm. Fitness is determined with suite of automatic evaluation metrics, and the Ludií i game description language also affords large number of semantic game "concepts" that are used to determine game novelty.
production of new artifacts) and computational (with the generation of new learning environments for artificial agents). Prior efforts in automated game design [42] 62] have produced some successes (notably the commercially--vaiill largely limited by hard-coded heuristics and restricted domains. These limitations are often necessary, however, in the face of automated game design's 's core challenges: (I) representing the vast array of possible games in a structured and computationally workable form and (2) efficiently searching through the resulting representation space for worthwhile games.
game Yavalath, generated by the Ludi system (12), but remain
In this work, we present GAVEL (Games via Evolution and Language Models)-an automated game design system that tackles these challenges by leveraging recent improvements in game rule representation and code synthesis (see overview in Figure T). GAVEL draws on three main components: (1) the Ludii game description language [6]66] to efficiently encode a large variety of board game rule sets, (2) a large code language model to reliably produce plausible modifications to existing games inspired by evolution through large models (ELM) [34], and (3) mmality-diversity
optimization [49] to generate a wide range of playable and interesting games. Each component builds on the others: our choice of representation allows GAVEL to not only produce novel board games in wide range genres and styles, but also affords us a dataset of over 1000 existing board games from around the world [10]. This dataset, in turn, provides sufficient basis fine-tune code synthesis model. In addition, our quality-diversin approach leverages the inherently modular nature of our representation in order to determine game novelty through the presence of particular game mechanics and motifs.
We show empirically that GAVEL is capable of generating playable and interesting board games that differ substantially from games encountered during training. Our approach intelligently recombines mechanics and ideas from disparate genres and produces samples that mirror the performance of suman-generatee games under a suite of automated evaluation metrics. preliminary qualitative analysis also reveals that GAVEL can generate novel games that are both engaging and entertaining. We conclude with a discussion of GAVEL'S 's successes and failures, as well as the promising avenues for future work. We provide public repository that includes our code and data, including trained model checkpoint
*Code and data available here: httpps://giie http://tttpptt//t/t/tttttccccttt

---

2 Related Work
2.1 Automatic Game Generation
Our work continues a strand of research that investigates the ability for automated systems to produce novel games or novel variants of existing games. The first such effort was METAGAME [45], which samples from a grammar that encodes "symmetric Chess-like games" with designer-specified rule probabilities. Since then, work has continued in the generation of both board game and video game rulesets. Evolutionary or search-based game design is a popular technique, as game descriptions do not typically afford gradient information; it first proposed in 2008 for board games [[2] and video games [62]. and later work has brought it to bear on different video game genres [43 [6 29]
Another approach begins instead from conceptual or symbolic specifications of rules mechanics and generates games by dynamically referring to a pre-specified library of gameplay elements 42 63)
Yet another approach is to use constraint satisfaction algorithms. For instance, rules might be encoded as an answer-set program, with constraints pre-specified by a designer to define what counts as an acceptable game [56]
26 [5]] Most recently. work has investigated the ability for large language models to act as design assistants by generating game levels [61, [58, proposing game mechanics
3 or directly synthesizing small programs 27
2.2 Evolutionary Computation and Language Models
Evolutionary computation refers to large class of algorithms broadly inspired by the biological process of evolution (24). Of these, our approach descends most directly from genetic programming: the use of evolution or other stochastic search procedures for program synthesis [88 23 32]. We
also draw more recent advances in quality-divessity algorithms that aim to find distribution of solutions to given problem rather than a single optima and content generation 23
20]. especially in the context code
In recent years, improvements in large language models both generally 7 70 and in their ability to produce code [14
67 have led to their use in variety of evolutionary systems. Of note is evolution through large models 34 which learns a diff model (i.e. a language model that can modify programs conditioned on natural language specification) from a dataset of GitHub commits and accompanying messages. This model then used as the mutation operator for genetic programming in Python through quality-diversity algorithm [41). We adopt this general approach, though GAVEL learns to mutate games from raw programs (i.c. without diffs or commit messages) in a domain-specifir language. Similar techniques have also been used to generate reinforcement learning environments
I 69 and reward functions (38), adversarial prompts [53 programming puzzles (48), and poetry 6
3 Game Representation and Dataset
We represent games as programs in the Ludii game description language (L-GDL) (11, in which game rules are built from siddmes--hggh-eewee keywords that represent common components in the natural language descriptions of board game rules. Examples of such keywords include s step, sl slide, hop, piece, empty. y, board, and so on. Owing to this abstraction, the L-GDL is both robust enough to encode a vast array of disparate games [46] and compact enough that game descriptions often fit within the context lengths of modern large language models.
In addition ludemes, the Ludi system also defines a large number of concepts- high level properties of games that describe its gameplay or structure [47] Most concepts are boolean and indicate the presence or absence of a particular feature. For example, one concept might describe whether game is asymmetric, while another might describe whether game uses the "custodial" capture mechanic seen in Tafl-style games. These concepts provide way to represent games as meaningful feature vectors, which can then be used to cluster and compute similarities between games [57].
3.1 Dataset
We construct our initial game dataset out of the 1182 existing games that have been translated into the Ludii i game description language (available under Creative Commons BY-NC-ND 4.0 license).
3

---

Cop Orpipman
1jous Bacid
Gueee FIRS sutat luak
Suace zurian luib
Oriiee
Crier
lid Que Galtes Remggtt
grlop Denve asl (siten Replu1311
Colop
1t (kae fil Sial Inites ansaad GaMr 1601
Oi Lume
Long
Contacted (oteen
Ocownn
Kinas Huit
Din Garernal (erwer))
Creads Noves SGA2
Figure 2: Left: the game of Havannah by Christian Freeling rendered in the Ludii i game description language. Center: the same game appears in the training dataset, with functional references expanded and game / piece names replaced with abstract identifiers. Right: variant of Havannah produced by GAVEL. Changes are highlighted yellow.
We start by expanding the function references (e.g. "BlockWin") inside each game description to the code they represent (e.g. (end (if (no Move Next) (result Mover Win) ) as specific functions may appear in only few games while the underlying ludemes are much more widespread. In addition, we remove references the particulars of each game (i.e. its name and the names of the pieces it uses) and replace them with abstract identifiers. Both these processes increase the generality of game description dataset.
After this, we filter our dataset to remove a small number of puzzles and experimental games. We also remove Mancala-style games, prior work has identified them as occupying very distinct cluster with respect to the full collection of Ludii i games, in terms of rules and structure [57]. We then tokenize each game according to our code language model (see below) and exclude any game that is longer than 1024 tokens. From this reduced dataset, we hold out a set 14 varied games (available in Anppendix A) that are used to initialize the evolutionary search, with the remaining 574 games being used as our training dataset. An example of one of the held-out games and its converted form is available in Figure 2)
4 Methods
4.1 Language Model Training
In line with the general ELM approach, we make use of the impressive generative capabilities of modern code language models order to propose sensible modifications existing programs [34]. Unlike prior work, however, we specifically fine-tune an existing model to operate in L-GDL instead of working with programming languages seen during pre-training or making use of in-context learning. In addition, we train our model to act mutation operator over programs by using fill-n-thhe-iddee (FITM) [5] training objective instead of the more common left-to-right objective. FITM training allows the model to make changes to interior components of program without (a) relying on an extant dataset of code diff's or (b) regenerating the entire game at each step.
We train an instance of CodeLl ama 52 (specifically CodeT.1 ama-133. as it is the largest model in its family that was pre-trained with FITM objective) on the dataset described in Seectinn 3.1 To facilitate FITM training, we extract every balanced parenthetical expression from each game (c.g. (board (square 10) )) and add the dataset along with the corresponding prefix and sulfix in the program. With respect to the grammar of L-GDL, this process is equivalent to extracting syntactic nodes and all of their descendants. The final dataset consists of 49,96 9,968 such (prefix, suffix, target) tuples. To facilitate training on single GPU, we make use of both
-fficiett fine-tuning (39] and 8-bit quantization [22]. Owing to the large size of the dataset and the fact that each game appears repeatedly in different configurations, we fine-tune the model for single epoch with hyperparameters available in Appendix B Training took approximately 40 hours to complete on single RTX8000 GPU.

---

We note here that FITM training may have a potential downside in the context of evolution through large models. Specifically, the model is trained to perfectly reproduce the missing section of code given its prefix and suffix. If it succeeds completely in doing so during evolution, then the resulting game will not be mutated at all. essence, there tension between the ability of the model to accurately capture the underlying logic and syntax of the representation space and its ability to memorize or perfectly reconstruct its training dataset. In GAVEL, we mostly sidestep this issue by mutating set of held-out games not seen at all during training (making memorization impossible), though see Appendix C for an initial investigation on mutating training-set games and [ecctinn 8 for a discussion of other possible approaches.
4.2 Evolutionary Search
Our evolutionary search strategy of choice is MAP-Elites [4], a population-based quality-diversin algorithm that leverages both a fitness function over samples as well as set of behavioral charac- teristics-functii functions that describe non-fitness attributes of samples and are used to ensure that the population does not collapse to only a small number of distinct samples. Specifically, MAP-Elitee maintains archive of cells, each associated with a particular range of values under the behavioral characteristics. At each step, a novel sample is evaluated to determine its fitness as well as the cell it would occupy. It is added to the archive if either that cell is unoceupied or if its fitness exceeds that of the current occupant, in which case it replaces the current occupant. In this way, samples only "compete" with one another within particular cells. We describe our fitness function in detail in Section 4.3
Determining appropriate set of behavioral characteristics is challenging in the context of automatic game generation. Intuitively, distinct archive cells ought to capture meaningfully distinct games while collapsing minor or trivial variations. Human game players readily make such categorizations, but automatically identifying such differences (especially with previously unseen games) is both difficult and necessary for the MAP-Elites algorithm to function. In order tackle this problem, we take advantage of the semantic concepts described in [Section 3] Building on evidence that Ludi principal component analysis (PCA) [66] on the complete Ludii dataset (i.e. before any filtering) to reduce the 510-dimensional concept vectors two dimensions. We then bucket the resulting two-dimensional space into 40 equally-spaced regions from -5 to 5 in each dimension, obtaining a rectangular archive of 1600 cells. While the first two PCA dimensions describe only 28% of the variance in the concept feature space, preliminary investigation indicated that increasing the number of dimensions resulted in less diverse archive (see [Appendix D for additional details). Because games are not uniformly distributed through feature space, increasing the archive's 's dimensionality for a fixed number of cells caused a larger number of distinct games to be mapped to the same cell. See discussion of potential alternatives in SSccion 7
concept vectors capture meaningful notion of distance [57],
We initialize the archive by adding and evaluating the 14 heldout games listed in Apppeddix A. For each MAP-Elites step, we select games from the current archive For each game, we then select k random parenthetical expressions and re-format them sample from the trained CodeLl ama- 1 133 model with a temperature of and a top-k value of 50 to generate a new expression, conditioned on just the prefix and suffix, and re-construct the resulting game. After filtering out any duplicate or unchanged games, the samples are evaluated for fitness and assigned an archive cell based on the PCA reduction of their concept vector.
(prefix, suffix, target) tuple. We then
4.3
Evaluation
Game quality is both difficult to quantify and inherently subjective. Nevertheless, automatic game design and evolutionary computation necessitate some kind of computable optimization objective. These objectives typically take the form of one more heuristics that aim to proxy the underlying targets of "fun" or "interestingness." For restricted domains, it is often possible to imbue a large amount of expert knowledge into these heuristics, as in the Ludi system [12 (precursor to Ludii) i) which employed stae- evaluator functions to capture both objective measures (e.g. game's 's balance) and psychological measures (e.g. a game's excitement or unpredictability). However, the large space of games described by the Ludi i description language makes relying on such evaluator functions infeasible. Instead, we define a hierarchical fitness function based on a relatively small set of objectively and reliably measurable heuristics that are fully game-agnostic.
5

---

Algorithm GAVEL Game Evaluation
Input: game g in L-GDL
Output: a fitness evaluation f(g) E (-3, -2, -1 [0.01,
if -ccppipiii (amle(() then
return
else if' mplayable(g) then
return -2
else
e random Saandom_eval(____ n 100)
if halance
< 0.5 or Eagency 0.5 then
return -1
else
V mcts_eveee (g. 10,4 10, d strategi ceattegicddeptiie c_depth 0.25, 50)
50)
f hmean Ubalance Edecisiveness: Ccompletion Vagency Veowerage d)
return f end
end if
Concretely, every game generated during the evolutionary search is assigned a fitness value f(0) E --,,-2 u [0.01, 1] by sequence of tests (pseudo-code presented in Algorithm 1). Evaluation begins with a series of binary evaluations. First, all games that fail to compile (c.g. due to grammatical errors, or because they do not define a board) are assigned the minimum possible fitness of -3 (we note that such games cannot be added to the archive as their concepts are not well defined). Next, all compilable games that cannot be played (e.g. due to failing to define any moves for pieces or failing to place pieces on the board for game like Chess) are assigned fitness of -2. If these conditions are cleared, then random-polle polliyy agents are used to rapidly obtain n 100 playouts of the game. If the difference in win rate between the first and second player is larger than threshold of 0.5 (indicating large imbalance even for completely naive players) fewer than half of game states allow more than one legal move (indicating lack of player agency), then the game is assigned a fitness -1. These checks are used as a filter to ensure that expensive evaluations are only performed on promising games.
If all three previous conditions are met, then search-based agents are used to obtain n 10 game playouts. Specifically, we use self-play between Monte-Carc Tree Search (MCTS) 31 17 agents with 0.25 seconds of thinking time per move and hard limit of = 50 moves per player any game which exceeds the limit is terminated and called draw. This limit puts strong pressure on games to end within a reasonable number moves and necessarily excludes many potentially interesting games (especially as search-limited agents might fail to find winning lines that could end game quickly). However, this concession necessary to ensure that the overall evaluation procedure remains computationally tractable.
From these 10 playouts, we extract the following evaluation metrics, largely inspired by prior work in automated game design 2 12]:
Balance: the largest difference in winrates between any pair of players.
2. Decisiveness: The proportion of games that do not end in draw.
3. Completion: the proportion of games that reach an end state.
4. Agency: the proportion of turns for which the player to move has more than one legal move.
5. Coverage: the proportion of board sites (e.g. squares on chessboard) that get occupied by
a game piece at least once in playout.
In addition, we separately compute final metric: Strategic Depth, defined the proportion of games won by an MCTS agent against random agent over n 10 playouts (and inspired by previous similar evaluations [4]
33 ). Each evaluation metric returns a value between 0 and 1,
and the metrics are then aggregated into single fitness score by taking the harmonic mean. We use
the harmonic mean over the simple average because the former weighted towards small values, penalizing games that succeed in most metrics but fail dramatically in one. We enforce minimum

---

All Cells
Novel Cells
Method
QD Score
# Playable
# Fitmess>0.5
# Playable
# Fitmess>0.5
GAVEL
395.62 = 17.46
117.67 9.46
106.67 = 7,41
26,67 3.30
21.67 6.13
GAVEL-UCB
341.17 = 14.39
96.67 = 6.02
88.33 + 7,41
19.67 : 4.03
16.00 : 2,45
Pure Sampling
296.92 # 14.84
89.00 5.66
83.00 5.10
14.67 3.30
11.33 2.87
GPT-40
268.16 # 17.33
84.67 = 6.60
80.67 # 5.19
16.67 # 3.30
15.33 # 2.87
Table I: Quantitative measures of archive progress for GAVEL, a variant in which mutation locations are selected with the UCB algorithm, and two baseline methods, averaged over three independent runs. We report the quality diversity (QD) score (a cumulative measure of fitness) as well as the number of archive cells and novel archive cells that reach certain fitness thresholds. Both GAVEL-based methods succeed in producing high-fitness games in unexplored regions of concept space, though CAVEL has the edge over GAVEL-UC3 in overall QD score. Compared to baseline methods, GAVEL achieves a significantly higher QD score and fills more of the archive will playable and high- fitness games.
Generation
Goneration 100
Gereration 200
Goneration 300
Gereratin 410
Generation 500
Lo
Figure 3: visualization of fitness of games generated by GAVEL over time. Starting from an initial archive of 14 games, GAVEL produced in this run 185 novel variations within 500 generations, of which 130 are playable and meet our minimum evaluation criteria. Further, 62 generated games occupy cells not covered by any game in the Ludii dataset and 29 of these games meet minimal criteria.
metric value of 0.01 before taking the harmonic mean ensure that the fitness score is not completely zeroed-out by single metric.
Experiments
We perform runs of MAP-Elites with random seeds (1, 2, 3), each lasting for 500 steps. For each run, we select j 3 games and generate k 3 mutations for each game at each step. Quantitatively, we report the progress of the archive using three metries: the quality-divesiity score (QD score) [50, calculated as the sum of the fitness of each cell in the archive. In cases like ours where fitness values can negative, each fitness value incremented by the minimum possible fitness (i.e. -2) before being summed to ensure that the QD score increases monotonically over time. In addition, report the number of playable and minimally interesting games (i.c. f(g) > 0) in the archive as well as the number of such games that occupy cells which not covered by any game in the Ludii i dataset. Finally, we report the number of cells and novel cells that contain games with a fitness of at least 0.5, indicating their potential as worthwhile games. Each run lasted roughly 48 hours using single RTX8000 GPU for inference from the Code ama- model and performing evaluations in parallel with 16 CPU cores and 128GB of total memory.
In addition, we perform another set of runs with variant of GAVEL that uses the Upper Confidence Bound algorithm 4 to select which regions of games to mutate. Specifically, we treat the selection of a parenthetical expression to mutate as a multi-armed bandit problem where each arm corresponds to different leading ludeme (c.g. board equi )..nt.. We consider mutation "successful" if it results in the mutated game being added to the archive (cither by improving the fitness an existing occupant, or by occupying new cell) and update the statistics for each "arm." We call this variant CAVEL-UCB. All other hyperparameters remain the same as with the original CAVEL experiment.
We compare GAVEL against two baselines: pure sampling approach, which uses the same fine-tuned large language model but omits the quality diversity search, and direct sampling from GPT-4o. For the pure sampling baseline, we randomly select game from the validation set and mutate it by re- generatertnng a random parenthetical expression. We repeat this process 4500 times in order to match the number of samples produced by GAVEL in 500 generations with j and & = 3. We then
7

---

evaluate the set of 4500 samples for fitness and determine the cell that each game would occupy based on its concepts, allowing us construct simulated archive (i.e. by retaining the highest-fitness sample in each cell) and compare directly against GAVEL, We perform the pure-sampling baseline experiment three times with random seeds {1, 2,3). For the GPT-4o baseline, we provide the model with each of the 14 validation games part of the context window and then ask it to create a modification of one randomly-selecten validation game (see) Jeeendpppiixx for prompt). We similarly generate 4500 samples and construct simulated archive in order to facilitate comparisons with GAVEL.
Results
6.1
Quantitative Results
Overall, GAVEL succeeds in generating wide range of novel and high-fitness games. In Table we present the mean and standard deviations of the archive metrics for GAVEL, GAVEL-UCB, and our baseline methods. The quality diversity score is difficult to interpret in isolation, as its magnitude depends greatly on the potential range of fitness scores. In this case, it is most helpful as a way to compare the performance of disparate algorithms on the same task: we see that GAVEL improves significantly over GAVEL-UCE (Welch's t-test, p 0.029), indicating that it has some mixture of higher fitness and greater variety in the samples it produces. Similarly, GAVEL improves significantly in terms of QD score over both the pure sampling baseline (p 0.004) and the GPT 4o baseline (p = 0.002).
Of more interest is the fact that CAVEL fills a substantial proportion of the archive with playable games despite starting from modest 14 samples, including in regions of the archive not covered by games in the Ludii i training dataset. Compared to both baselines, GAVEL occupies significantly more cells in the archive and produces more high-fitness (i.e. [(g) > 0.5) samples (p < 0.03 for pure sampling, p < 0.02 for GPT-4o), though the differences between GAVEL and GAVEL-UCB are not significant (p > 0.05). While GAVEL also appears to occupy larger number of novel cells with playable and high-fitness games compared to the basclines, these improvements are not statistically significant at only three runs. We present a visualization of the archive produced by one run of GAVEL over time in Figure 3 which shows both the success of the model in generating high-fitness samples and the fact that much of the concept space remains unexplored.
6.2 Qualitative Results
In order to more closely examine GAVEL'S output, we rely on expert evaluators quickly playtest potentially promising games. These evaluators are broadly familiar with the Ludi dataset and so are able to determine whether generated game is truly novel and where its mechanics might have originated from. This preliminary human analysis helped shed light on some of GAVEL's shortcomings (discussed below) and also revealed some particularly interesting games among the high-fitness samples. Of special note is a variant of Yavalath, itself the product of an automated system [1]. In the original game, players take turns placing pieces on hexagonal board. player wins if they have four pieces in row but loses if they have three pieces in a row first. CAVEL makes both minor changes to the ending rules (increasing the number of pieces in a row needed for victory and loss by one) well substantial addition by introducing the enclosure capture rules of Go. The result a game that tasks players with thinking about the arrangement of their pieces in many ways and that appears to offer the potential for sophisticated strategy. In Figure 4 we present an example of play between automated agents in which all of the game's 's rules are used in concert.
Figure 2 (right) includes another example of a generated game noted by our evaluators to be partic- ularly interesting. It is a variant of Havannah (a game which players attempt to form loops or connected lines of pieces between sides of the board) that introduces a restriction on piece placement from another game in the Ludii dataset (Tabu Y). A final exemplar, presented in /Appendix F alongside the previous two examples, modifies the pawn-ad wwwn--ddannemmeen game Breakthrough to use pieces that can only move by hopping over each other (and without capturing). Taken together, these examples demonstrate the strength of GAVEL: it is able to intelligently recombine game mechanics (expressed as code segments) novel ways and ensure that these combinations do not result in trivial games.
8

---

Cpaoy "TaraGot (glayara
(egutpmant
Dound
8333
(stace "Narzer"" Each)
33
Repeat)
(play
(cove Add
Other
(co (atsen tspsy))
1. What seems to be relatively balanced postion...
2. Black moves, capturing and setting
(encicoe
Orthegina) Cronovo (botuena33)
(betswwa)))
(ond
CLE (ix Lise Cremeit Sext Loua) SLT (ie. Line Creealt Soat Win33
Players tske tume placing markers on hexagonsl board. Getting 5 pieces in row wins, but getting 4 row loses. Surrounding your opponent's pieces
only possible play to not lose on the spot...
singlo move, black removes while's diffnns and sets throat
Figure 4: Examplc of play between MCTS agents in game generated by GAVEL. The game descended from Yavalath (an n--in-a-row style game) and combines modification of its ending rules with the enclosure capture mechanics of Go. Search-based agents reach interesting and strategically deep game positions, hinting its potential interest to human players as well.
7 Discussion and Limitations
Unused Game Components: common failure mode is that the model will generate game rules that are not actually used during gameplay. For instance, a change to the equipment section might add dice additional game pieces. However, without further changes to the gameplay section to incorprate them, the dice will remain unused. Detecting such extraneous rules automatically is challenging, the Ludii system does not provide way determine which rules are activated during a given playout. evaluation might harm diversity by eliminating potential "stepping stones" more interesting games. Nevertheless, if changes to the underlying representation did allow unused sections to be detected, might be possible to bias future mutations towards relevant gameplay sections in order increase the likelihood of the missing rules being generated.
addition, penalizing games with unused components during fitness
Heldout Games: noted in Section 4.2 we initialize the search using set of games held out from the language model training in order to prevent memorization. While GAVEL produces a wide range of games from this set, they nonetheless will share many features with another result of their common origins. One potential solution to further increase archive diversity is to improve the variety of mutations, either by increasing sampling temperature or by enforcing novelty with respect the tokens the original game section through masking. These techniques might also make it possible to initialize the archive with games in the training dataset, further increasing potential output diversity.
Archive Selection: determining an appropriate way to distinguish between games is a general challenge. GAVEL makes use of Ludii concepts, but not all representation schemes afford such detailed semantic information. One promising alternative for such domains is the automatic selection of behavioral characteristics, either through distillation of trajectories obtained during evaluation 19] or by leveraging the ability for large language models to identify archetypes in code 48
Evaluation: our evaluation metrics capture general and minimal criteria of interesting games, broadly construed. However, satisfying our evaluation metrics alone is from sufficient evidence for a game being interesting. Ultimately, all metrics in automated game design aim to proxy notions of human preference- as mentioned in [Section 6.2] we rely on expert evaluators to filter from high-fitness samples interesting games. Another worthwhile approach, then, might be to learn these preferences

---

directly from human ratings [5] models 30
or attempt to extract them from latent knowledge in large language
8 Future Work
First and foremost, we are excited about a more in-depth human analysis of the games generated by GAVEL. While our expert evaluators can provide useful insight and analysis, their perspective ultimately one of many. A larger user study could help identify not only which games are most appealing to human players, but also the particular features of those games that correlate with fun and engagement. This, in turn, could help spur improvements to GAVEL across the board, from archive selection to evaluation metries.
Within the Ludii domain, one particularly exciting possibility for future work the integration of the explicit L-GDL grammar. While the Code aaae-lmmmm20 ama-1ab model learns to produce syntactically valid mutations, integrating the grammar either through token masking [54] or Monte-Carlo steering 35 could allow for greater mutation diversity (by increasing sampling temperature, for instance) without sacrificing syntactie correctness and compilability,
addition, it might be possible to use aata-augmentttion techniques (e.g. sampling from the underlying grammar) to create a dataset of Ludii game diffs with which to train a more typical ELM model.
More generally, large language models could also be used to link natural language descriptions of game rules with their programmatic representations. For example, an instruction-tuned model could be used to convert from abstract rules to executable code, either by fine-tuning (e.g. on the Ludi dataset) or through in-context learning. This might allow automatic game design systems to interact with and generate games at the level of natural language, better resembling the process used by human designers, while still retaining the ability to automatically evaluate the relevant gameplay properties of those games.
Finally, our results indicate that systems like GAVEL might be most useful in the context of co- creativity [21] 68). Automated processes are able to rapidly generate plausible combinations of game mechanics, while human designers and play-testers are able much better determine the subtle changes necessary to elevate a potentially interesting idea to an entertaining and engaging game. A system which explicitly integrates such expertise may thus prove to be the best way forward.
Broader Impact
Like all automated game design systems, GAVEL has the potential for impacts on the larger space of game design. Especially at time of writing, the video game industry experiences widespread layoffs and contractions, it is important that automatic systems are used to assist and inspire human designers instead of replacing them. Indeed, our results indicate that such collaboration is crucial to the generation and identification of worthwhile games. We also draw attention to GAVEL's 's use of large language models: while the Ludi i dataset is publicly available, similar system could conceivably be used to generate games from scraped datasets without such free access or from game code encountered during pre-training. Special care should also be taken to ensure that language model outputs are manually verified before being published. Overall, however, we feel that the specific domain, use case, and outputs of GAVEL mean that its broader impact is unlikely be negative.
Acknowledgments
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant DGE-2234660, This research is partially funded by the European Research Council as part of the Digital Ludeme Project (ERC Consolidator Grant #771292) The research collaboration was facilitated by COST Action CA22145 GameTTable, supported by COST (European Cooperation in Science and Technology). The authors would like thank Rushang Gajjal and Sam Earle for their help and constructive feedback throughout the project.
10

---

References
[1] F. Aki, R. Ikeda, T. Saito, C. Regan, and M. Oka. LIm-poet: Evolving complex environments using large language models. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 243-246 2024.
[2] I. Althöfer. Computer-Aided Game Inventing. Technical report, Friedrich-Schiller University Jena, Faculty of Mathematics and Computer Science, Jena, Germany, 2003.
[3] A. Anjum, Y. Li, N. Law, and J. Togelius. The ink splotch effect: A case study chatgpt as a co-creative game designer. In Proceedings of the 19th International Conference on the Foundations of Digital Games, pages 1-15, 2024.
[4] P. Auer. Using confidence bounds for exploitation-explratiio trade-offs. Journal of Machine Learning Research, 3(Nov):397-422 2002.
[5] M. Bavarian, H. Jun, N. Tezak, J. Schulman, C. McLeavey, J. Tworek, and M. Chen. Efficient training of language models to fill in the middle. arXiv preprini arXiv:2207. 2022.
[6] H. Bradley, Dai, H. Teufel, J. Zhang. Oostermeijer, M. Bellagente, J. Clune, K. Stanley,
G. Schott, and J. Lehman. Quality-diversity through ai feedback, 2023.
[7] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam,
G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1900 2020.
[8] C. Browne. Quickly detecting skill trace in games. In Proceedings the
the 2022 IEEE Conference
on Games, pages 604-607, 2022.
[9] C. Browne, E. Powley, D. Whitehouse, S. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton. A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games, 4(1):1-49, 2012.
[10] C. Browne, D. J. J. J. Soemers,
Piette, M. Stephenson, M. Conrad, W. Crist, T. Depaulis, E. Duggan, Horn, S. Kelk, S. M. Lucas, P. Neto, D. Parlett, A. Saffidine, U. Schädler, J. N. Silva, A. de Voogt, and M. H. M. Winands. Foundations of Digital Archæooudology. Technical report, Schloss Dagstuhl Research Meeting, Germany, 2019.
[11] C. Browne, D. J. N. J. Soemers, É. Piette, M. Stephenson, and W. Crist. Ludii language reference. [udii game downloads /tt///Ldiiiiii Lawaaaaaaaaaaaaaueeeeeeeeeennnnn pdf 2020.
[12] C. B. Browne. Automatic Generation and Evaluation of Recombination Games. Phd thesis, Fac- ulty of Information Technology, Queensland University of Technology, Queensland, Australia, 2009.
[13] M. Campbell, A. J. Hoane Jr., and F. Hsu. Deep Blue. Artificial Intelligence, 134(1 134(1-2):57-83, 2002.
[14] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto. J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2 2021.
[15] W. Chu and Z. Ghahramani. Preference learning with gaussian processes. In Proceedings of the 22nd international conference Machine learning, pages 137-144, 2005.
[16] M. Cook, S. Colton, and J. Gow. The angelina videogame design system part i. IEEE Transactions on Computational Intelligence and Al Games, 9(2):192-203, 2016.
[17] R. Coulom. Efficient selectivity and backup operators in Monte-Cari tree search. In H. J. van den Herik, P. Ciancarini, and H. H. L. M. Donkers, editors, Computers and Games, volume 4630 of Lecture Notes in Computer Science, pages 72-83. Springer Berlin Heidelberg, 2007.
[18] N. L. Cramer. A representation for the adaptive generation of simple sequential programs. In Proceedings of the First International Conference on Genetic Algorithuss and Their Applications: July 24-26, 1985 at the Carnegie-Mellon University Pittsburgh, PA, pages 1 183-187. Psychology Press, 1985.
[19] A. Cully. Autonomous skill discovery with quality-diversity and unsupervised descriptors. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 81-89, 2019.
[20] A. Cully and Y. Demiris. Quality and diversity optimization: unifying modular framework. IEEE Transactions on Evolutionary Computation, 22(2):245-259, 2017.

---

[21] N. Davis. Human-compp computer co-creativity: Blending human and computational creativity, In Pro- ceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment volume 9, pages 9-12, 2013.
[22] T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer. Gpt3. int8 (): 8-bit matrix multiplication for transformers scale. Advances in Neural Information Processing Systems, 2022.
[23] D. Dickmanns, Schmidhuber, and A. Winklhofer. Der genetische algorithmus: Eine im- plementierung in prolog. Fortgeschritteennpaakiiia Institut f ur Informatik, Lehrstuhl Prof. Radig, Technische Universit M unchen, 1987.
[24] A. E. Eiben and J. E. Smith. Introduction to evolutionary computing. Springer, 2015.
[25] D. Gravina, A. Khalifa, A. Liapis, J. Togelius, and G. N. Yannakakis. Procedural content generation through quality diversity. In 2019 IEEE Conference on Games (CoG), pages 1-8. IEEE, 2019.
[26] M. Guzdial and M. Riedl. Automated game design via conceptual expansion. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, volume 14, pages 31-37, 2018.
[27] C. Hu. Y. Zhao, and J. Liu. Generating games via Ilms: An investigation with video game description language, 2024.
[28] E. J. Hu, P. Wallis, Z. Allen-Zhu, Li, S. Wang, L. Wang, W. Chen, al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021.
[29] A. Khalifa, M. C. Green, D. Perez-Liebana, and J. Togelius. General video game rule generation. In 2017 IEEE Conference on Computational Intelligence and Games (CIG), pages 170-177 IEEE, 2017.
[30] M. Klissarov, P. D'Oro, S. Sodhani, R. Raileanu, P.-L. Bacon, P. Vincent, A. Zhang, and M. Henaff. Motif: Intrinsic motivation from artificial intelligence feedback. arXiv preprint arXiv:2310.00166 2023.
[31] L. Kocsis and C. Szepesvári. Bandit based Monte-Carlo planning. In Fürnkranz, T. Scheffer, and M. Spiliopoulou, editors, Machine Learning: ECML 2006, volume 4212 of Lecture Notes in Computer Science, pages 282-293. Springer, Berlin, Heidelberg, 2006.
[32] J. R. Koza. Genetic programming: paradigm for genetically breeding populations of computer programs to solve problems, volume 34. Stanford University, Department of Computer Science Stanford, CA, 1990.
[33] F. Lantz, A. Isaksen, A. Jaffe, A. Nealen, and J. Togelius. Depth in strategic games. In Workshops at the Thirty- First AAAI Conference on Artificial Intelligence, 2017.
[34] J. Lehman, J. Gordon, S. Jain, K. Ndousse, C. Yeh, and K. O. Stanley. Evolution through large models. In Handbook of Evolutionary Machine Learning, pages 331-366. Springer, 2023.
[35] A. K. Lew, large language models using probabilistic programs. arXiv preprint arXiv:2306.0308) 03081, 2023,
G. Grand, and V. K. Mansinghka. Sequential monte carlo steering of
[36] C. Linnnuus. Iter Lapponicum. Stockholm, Sweden, 1732.
[37] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint 17Xiv: 101, 2017.
[38] Y. J. Ma. W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y. Zhu, Fan, and
A. Anandkumar. Eureka: Human-level reward design via coding large language models. In The Twelfih International Conference on Learning Representations, 2024.
[39] S. Mangrulkar, S. Gugger, L. Debut, Y. Belkada. S. Paul, and B. Bossan. Peft: State-of-ther art parameter-efficine fine-tuning methods. httpp::/ /ttt:::///ithbbbbb com www.///wgggggggggc.ecccoooo ft 2022.
[40] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. Human-level control through deep rein- forcement learning. nature, 518(7540):529-33, 2015.
[41] J.-B. Mouret and Clune. Illuminating search spaces by mapping elites. arXiv preprint 1rXiv:1504.04909 2015.
12

---

[42] M. J. Nelson and M. Mateas. Towards automated game design. In AI IA 2007: Artificial Intelligence and Human- Oriented Computing: 10th Congress of the Italian Association for Artificial Intelligence, Rome, Italy, September 10-13, 2007. Proceedings 10, pages 626-637. Springer, 2007.
[43] T. S. Nielsen, G. A. Barros, J. Togelius, and M. J. Nelson. Towards generating arcade game rules with vgdl. In 2015 IEEE Conference on Computational Intelligence and Games (CIG), pages 185-192. IEEE, 2015.
[44] J. C. Osbom,, A. Summerville, and M. Mateas. Automated game design learning. In 2017 IEEE Conference on Computational Intelligence and Games (CIG), pages 240-247. IEEE, 2017.
[45] B. Pell. Metagame in symmetric chess-like games. University of Cambridge Computer Laboratory. 1992.
[46] É. Piette, D. J.
J. Soemers, M. Stephenson, C. F. Sironi, M. H. M. Winands, and C. Browne. Ludii - the ludemic general game system. In G. D. Giacomo, A. Catala, B. Dilkina, M. Milano, S. Barro, A. Bugarín, and J. Lang. editors, Proceedings of the 24th European Conference on Artificial Intelligence (ECA/ 2020), volume 325 of Frontiers in Artificial Intelligence and Applications, pages 41 1-1..18. IOS Press, 2020.
[47] É. Piette. M. Stephenson, D. J. J. Soemers, and C. Browne. General board game concepts. In Proceedings of the 2021 IEEE Conference on Games (CoG), pages 932-939, IEEE, 2021.
[48] J. Pourcel, C. Colas, P.-Y. Oudeyer, and L. Teodorescu. Aces: generating diverse pro- gramming puzzles with autotelic language models and semantic descriptors. arXiv preprint arXiv:2310. 10692, 2023.
[49] J. K. Pugh, I.. B. Soros, and K. O. Stanley. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, 3:202845, 2016.
[50] J. K. Pugh. L. B. Soros, P. A. Szerlip, and K. O. Stanley. Confronning the challenge of quality diversity. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pages 967-974, 2015.
[51] T. Rolle. Development of a multi-game engine. Diploma thesis, Friedrich-Schiller University Jena, Faculty of Mathematics and Computer Science, Jena, Germany, 2003.
[52] B. Roziere, Gehring, Gloeckle, S. Sootla, Gat, X. E. Tan, Y. Adi, Liu, T. Remez,
J. Rapin, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308. 2023.
[53] M. Samvelyan, S. C. Raparthy, A. Lupu, E. Hambro, A. H. Markosyan, M. Bhatt, Y. Mao, M. Jiang, J. Parker-Holder, J. Foerster, et al. Rainbow teaming: Open-ended generation of diverse adversarial prompts. arXiv preprint arXiv:2402. 16822, 2024.
[54] T. Scholak, N. Schucher, and D. Bahdanau. Picard: Parsing incrementally for constrained auto-regressive decoding from language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9895-9901, 2021.
[55] D. Silver, A. Huang, C. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser,
I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalch- brenner, I. Sutskever, Lillicrap, M. Leach, K. Kavukeuoglu, T. Graepel, and D. Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):884- 489, 2016
[56] A. M. Smith and M. Mateas. Variations forever: Flexibly generating rulesets from a sculptable design space of manis. In Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games, pages 273-280. IEEE, 2010.
[57] M. Stephenson, D. N. Soemers, É. Piette, and C. Browne. Measuring board game distance. In C. Browne, A. Kishimoto, and J. Schaeffer, editors, Computers and Games. CG 2022, volume 13865 of Lecture Notes in Computer Science, pages 121-130. Springer, Cham, 2023.
[58] S. Sudhakaran, M. Monzález-Duquee M. Freiberger, C. Glanois, F. Najarro, and S. Risi. Mari- ogpt: Open-ended text2level generation through large language models. Advances in Neural Information Processing Systems, 36, 2024.
13

---

[59] A. Summerville, C. Martens, B. Samuel, J. Osborn, N. Wardrip- Frrii, and M. Mateas. Gemini: Bidirectional generation and analysis of games via asp. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, volume 14, pages 123-129, 2018.
[60] B. Tjanaka, M. C. Fontaine, D. H. Lee, Y. Zhang, N. R. Balam, N. Dennler, S. S. Garlanka, N. D. Klapsis, and S. Nikolaidis. Pyribs: A bare-bones python library for quality diversity optimization. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '23, page 220-229, New York, NY, USA, 2023. Association for Computing Machinery.
[61] G. Todd, S. Earle, M. U. Nasir, M. C. Green, and J. Togelius. Level generation through large language models. In Proceedings of the 18th International Conference the Foundations of Digital Games, pages 1-8, 2023.
[62] Togelius and Schmidhuber. An experiment in automatic game design. In 2008 IEEE Symposium On Computational Intelligence and Games, pages 111-118. IEEE, 2008.
[63] M. Treanor, B. Blackford, M. Mateas, and I. Bogost. Game-o-matic: Generating videogames that represent ideas. Proceedings of the The third workshop on Procedural Content Generation in Games, pages 1 1-8, 2012.
[64] V. Vassiliades, K. Chatzilygeroudis, and J.-B. Mouret. Using centroidal voronoi tessellations to scale up the multidimensional archive of phenotypic elites algorithm. IEEE Transactions on Evolutionary Computation, 22(4):623-630, 2017.
[65] O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, J. Oh, D. Horgan, M. Kroiss, I. Danihelka, A. Huang, L. Sifre, Cai, J. P. Agapiou, M. Jaderberg, A. S. Vezhnevets, R. Leblond, T. Pohlen, V. Dalibard, D. Budden, Y. Sulsky, J. Molloy, T. L. Paine, C. Gulcehre, Z. Wang, T. Pfaff, Y. Wu, R. Ring, D. Yogatama, D. Wünsch, K. McKinney, O. Smith, T. Schaul, T. Lillicrap, K. Kavukcuoglu,
D. Hassabis, C. Apps, and D. Silver. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575:350-354, 2019.
[66] S. Wold, K. Esbensen, and P. Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1 2(1-3):37-52, 1987.
[67] F. F. Xu, U. Alon, G. Neubig, and V. J. Hellendoom. A systematic evaluation of large language models of code. Proceedings of the 6th ACM SIGPIAN International Symposium on Machine Programming, pages 1-10, 2022.
[68] G. N. Yannakakis, A. Liapis, and C. Alexopoulos. Mixed-initiative co-creativity. Foundations of Digital Games, 2014.
[69] A. Zala#, J. Cho* H. Lin, J. Yoon, and M. Bansal. Envgen: Generating and adapting environ- ments via 11ms for training embodied agents. In COLM, 2024.
[70] W. X. Zhao, K. Zhou, Li, Tang, X. Wang, Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong,
et al. A survey of large language models. arXiv preprint arXiv:2303.. 18223, 2023.
[71] A. Zook and M. Riedl, Automatic game design via mechanic generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
14

---

Appendices
A
Held-Out Games
The following games were removed from the Ludii i training dataset and used initialize the MAP. Elites archive in each run, each of which are available in the Ludii portal (https: //ludii
games / wwt.rrrrbbrrppp..........
Ard-Ri (traditional Scottish game)
Ataxx (Dave Crummack & Craig Galley, 1990)
Breakthruugh (Dan Troyka, 2000)
Gomoku (traditional Japanese game)
Havannah (Christian Freeling, 1981)
Hex (Piet Hein, 1942)
Knightthrouhh (probable origin: http://games.ggeeeee ogpp://..aaa............
Konane (traditional Hawaïann game)
Pretwa (traditional Indian game)
Reversi / Othello (Lewis Waterman John W. Mollet, 1883)
Shobu (Manolis Vranas & Jamie Sajdak, 2019)
Tablut (traditional Finnish game 36
Tron (probable origin:
ggp..rg...........
Yavalath (Ludi system, 2009)
B Language Model Training Hyperparameters
The instance of Codel1 ama- 3b used by GAVEL trained with the following hyperparameters:
Number of epochs:
Batch size:
Sequence length: 1024
Optimizer: Adami [37
Learning rate: 3 10"
Warmup Ratio: 0.03
In addition, the model was trained with low-rank adaptation (LoRA) 28 and the following LoRA- specific hyperparameters:
LoRA Alpha: 16
LORA Dropout: 0.05
LoRA r: 64
15

---

C Effects of Mutating Training Games
We present a preliminary investigation on the feasibility of initializing GAVEL with games from the training dataset rather than the held-out validation set. Using the same fine-tuned CodeL ama- language model, we generate 300 mutations by sampling game from the training set, sampling a parenthetical expression from the game, and generating 3 replacements from the model. We perform the same process for the validation set, producing another 300 mutated games. For each mutation, we then compute whether it is an exact duplicate of the originally-sample game (i.e. whether it is novel) and whether the resulting game compiles under the Ludii i grammar (i.e. whether it is valid). We repeat the entire process for different sampling temperatures and settings of k for top-k sampling, the results of which are presented in Table 2
Unsurprisingly, mutations produced from training-ser -ee games are much less likely to be novel (since the model may have memorized the exact target sequence during training). The fact that roughly half of validation-set mutations are also duplicates might seem surprising at first, but it is important to note that many sampled parenthetical expressions are very short (e.g. (sites) ) and might be the only grammatical possibility at that point the game. Training-set mutations do appear to be compilable more often than validation-set mutations (especially at higher sampling temperatures), but this is likely due at least in part to the increased rates of duplication. The proportion of samples that are both novel and valid (i.e. those that represent potentially useful mutations) tells a clear story: sampling mutations from validation-set games is more efficient than doing so from training-se set games. A final point of interest is that increasing sampling temperature appears to improve the efficacy of training-set sampling while decreasing the efficacy of validation-set sampling. This further motivates the possible extensions deseribed in Section 7 and [Section 8] high sampling temperature coupled with grammatical sampling constraints might make training-set mutations viable way forward.
Training-set Mutations Valid
Validation-sel Mutations Valid
Temperature
Top-K
Novel
Novel & Valid
Novel
Novel & Valid
20
17.33
99.33
16.67
47.33
98.67
46.00
0.5
50
17.33
99.33
16.67
47.33
98.67
46.00
20
21.00
97.33
18.33
50.67
93.67
44.33
1
50
21.00
97.33
18.33
49.00
94.00
43.00
20
33.67
88.67
22.33
56.67
76.67
33.33
1.5
50
35.66
85.33
21.00
54.33
73.67
28.33
Table 2: The proportion of mutations generated from training-set and validation-set games that are novel, compilable, and both novel and compilable. We see that training-set mutations often duplicate the original game, a tendency which is not especially ameliorated by higher sampling temperatures.
D Effects of Archive Dimensionality and Size
As part of initial experiments for GAVEL, we explored the effect of changing the number of PCA dimensions and the total number of cells used by the MAP-Elites archive on the number unique cells occupied by training and validation games. In each case, we fit a PCA model with specified number of components on the Ludií dataset using the process described in Seetionn 4.2] For a target number of total archive cells C' and number of dimensions D, constructed a rectangular archive by separating each axis from -5 to into (((1/)) evenly-spaced regions. Of course, for many choices of C' and D, this process produces an archive with a total size much less than C' (e.g. for D 4 and C = 1000, the archive would have 5 625 cells). To combat this, we increased the number of regions in the first dimension as much as possible while keeping the total number of cells no more than C (e.g. for D= 4 and C 1000, the number of regions in each dimension is [8, 5, 5, 5] for total archive size of 1000).
We then simulated adding each of the 574 training games and 14 validation games from our dataset into the archive and measured the number of unique resulting cells. This gives a sense of the "diversity" of the archive if only a small number of cells are occupied it indicates that the archive fails to
16

---

distinguish between substantively distinct games. On the other hand, if the number of occupied cells roughly equal to the number of games, this might indicate that the archive assigns even almost identical games to different cells and could slow the search process. The results of these experiments are presented in Table Our investigation indicated that increasing the dimensionality for fixed target archive size caused the set of training games to be collapsed to smaller set of cells a 2-dimensional archive with 2500 cells is roughly as diverse under this formulation as a 3-dimensional archive with 10000 cells. We ultimately decided to use 2-dimensional archive with roughly 1500 cells because it was the smallest archive that mapped each of the validation games to distinct cells.
Target total archive cells
Input set
Dimension
100
500
1000
1500
2500
5000
10000
32
105
166
215
268
370
410
3D
24
59
74
101
137
191
247
Training (n 574)
4D
19
52
75
109
128
180
5D
45
32
48
70
99
98
126
2D
13
14
14
14
3D
6
8
11
11
10
13
Validation (n 14)
4D
5
7
10
9
12
5D
7
7
8
9
8
Table 3: The number of archive cells occupied by the 574 training set games and 14 validation set games, based the dimensionality and larget total number of cells in the archive. We that increasing the dimensionality for a fixed target archive size causes set training games to collapsed to smaller set cells and that 2-dimensional archive is the only one that maps each of the validation games to unique cell.
We also briefly explored the possibility of using Centroidal Voronoi Tesselation (CVT) based approach (64] to combat the high dimensionality of the full concept vector space. The CVT archive is designed scale nigh-dimensional problems by breaking down the search space into pre-specified number of geometrically homogeneous niches, so the full 5 510-dimensional Ludii concept vectors using the implementation in the Pyribs library [60) and thereby avoid the need for lossy dimensionality reduction. We set the boundaries in each dimension [O. 1] because each concept is binary. Unfortunately, even very large archives (i.e. with 100000 cells) collapsed the set of training and validation games into a small number of unique cells. We attribute this the fact that Ludii games are uniformly distributed throughout the space of concepts (i.e. many concepts are correlated or mutually exclusive) while the CVT algorithm assigns equal "resolution" all parts of the search space. However, it possible that some combination of PCA and the CVT archive might achieve better balance of archive size and diversity.
sophisticated kind of archive (namely
we attempted to apply
Number of archive cells
Input Set
100
500
1000
1500
2500
5000
10000
50000
100000
Training (n 574)
19
48
36
37
41
68
Validation (n = 14)
3
8
7
9
7
Table 4: The number of CVT archive cells occupied by the 574 training set and 14 validation set games, based on the total number of cells in the archive. We see that even very large archives collapse the input sets to small number of cells.
17

---

E LLM Baseline Details
As mentioned in Sncciion 5| we use the GPT language model through the OpenAI API. To increase the diversity of generated samples, we use sampling temperature of 1. We specify unique seed for each sample from the model (i.e. seeds 0 through 4499 for the first experiment), though the OpenAl API does not guarantee reproducibility for a specific seed. We provide the system prompt and user prompt used baseline experiments below.
System Prompt
You are expert programming agent in the Ludii game descripticn language, You output syntactically correct Ludii game descriptions and no other text of any kind.
Game Modification Prompt
Your task is to mutate game written in the Ludií game description language to produce game. Use the following games as reference for proper Ludii syntax and game structure:
cam==Game l=ss=== REFEERNNE GAME CODE:
===== Game i=ss===
REFERENCE GAME CODE
Now, create modi modification of the following game, Make sure to obey the constraints of the Ludii grammar to create a syntactically-valid games. In addition, make sure to modify at least part of the game that it becomes new game. Do not simply copy an existing game.
=e===Game to modify===e.
GAME CODE TO MODIFY:
Modified game
18

---

GAVEL Exemplars
Havabu
This game is variant of Havannah. Players take turns placing pieces a square board. If player manages form line of pieces (including diagonals) that connects three different sides of the board, they win. For the purposes of this condition, corners of the board do not count as belonging to any side however, connecting two corners also results in win. Finally, player loses if they do not have any valid moves. Players are also restricted in that they cannot place piece any of the 8 squares adjacent the previous move. It is available to play here: https 7/ludii. gates//amaaa//eeeee http//teaaallll.e..m.. php Reyword Havaa
(game "Havabu"
(players 2)
(equipment {
(board (square 8))
(piece "Marker" Each)
}) (rules
(play
(move Add (to (sites Empty)
if: (not (is In (to) (sites Around (last To) ))
(end
(if
(or {
(no Moves Next)
(is Connected SidesNoCorners)
(is Connected Corners)
})
(result Mover Win)
19

---

YavaGo
This game is variant of Yavalath. Players take turn placing pieces on hexagonal grid. player gets five pieces in row, they win. However, if player gets four pieces in row, they lose (meaning that five-in-a-row lines must be constructed out of two smaller pieces). In addition, completely surrounding one more of your opponent's pieces causes them to be removed from the board. It is available to play here: https://ludi games/detaills php.keyoord=yaaae
(game "YavaGo"
(players 2)
(equipment {
(board (rotate 90 (hex )
(piece "Marker" Each)
(rules
(meta (no Repeat))
(play
(move Add
(to (sites Empty))
(then
(enclose
(from (last To) Orthogonal
(between if: (is Enemy (who at: (between)) )
(apply (remove (between))
(end {
(if (is Line 5) (result Next Loss)
(if (is Line 4) (result Next Win)
)
20

---

HopTTroough
This game is a variant of Breakthrough Knightthrouh.. Players start with two rows of pieces on opposite sides of the board, and the objective is to have a piece reach the other end. Pieces can only move by hopping over one another (either laterally diagonally), and there are no captures. is available to play here: https://ludi: games/detai 1s. phpp
(game "HopThrough" (players 2)
(equipment {
(board (square 8))
(piece "Counter" Each
(move Hop
(between if: (is Occupied (between) )
(to if: (is Empty (to)))
(regions P1 (sites Top))
(regions P2 (sites Bottom)
})
(rules
(start {
(place "Counter1" (expand (sites Bottom)) )
(place "Counter2" (expand (sites Top)))
)
(play
(forEach Piece)
(end
(if (is In (last To) (sites Mover)) ) (result Mover Win)
21

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The claims the introduction and abstract concern the successful generation of novel and interesting games and evidenced by quantitative and qualitative results later in the paper.
Justification:
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The paper includes dedicated Limitations section that discusses the limita- tions of the work (Section 7)
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper contains the details necessary to train the LLM used as mutation operator, well the specifics of the MAP-Elites algorithm and evaluations used during experiments. In addition, we are in the process of anonymizing and releasing a saved model checkpoint.
Justification:
5. Open access to data and code
Question: Does the paper provide open to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide link to public repository that includes our data and code, including a trained model checkpoint, as a footnote at the end the introduction and here:
https: https://github com/ gdrt ood/gavel
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: These details are available in Section 3.1
Section 4.1
Appendix A and
Appendix
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Quantitative results include reports standard deviation, and significance testing used when comparing results between experimental conditions.
22

---

8. Experiments Compute Resourees
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: These details are reported in Section 4.1 and Section 5
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics wtt.pp/tt.e.//auu.....oooooooota
Answer: [Yes]
Justification: The authors have reviewed the Code of Ethies and affirm that the paper conforms to its specifications.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The potential impacts of the work are discussed in Section
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The language model used in the paper poses risks beyond its original pre-trained form, and no other components the system have high risk for misuse.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
Both the model and dataset are properly credited and are used with appropriate licenses.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce new assets.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justificattin: The paper does not involve crowdsourcing nor research with human subjects. 15. Institutional Review Board (IRB) Approvals Equivalent for Research with Human
Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
23


Reviews:
[{"review_id": "fN74jW7vNY", "reviewer": "Reviewer_29DD", "summary": "The authors consider the problem of generating sets of diverse and interesting multi-player games. They instantiate this problem in the subspace of board-game like games using a recent domain-specific language called Ludii. They then use a quality-diversity algorithm (specifically, map-elites with a language-model-based mutation operator) as their generation algorithm. They name their overall approach GAVEL.\n\nFor the mutation operator, the authors fine-tune a 13b codellama model with a fill-in-middle objective to reconstruct structured portions of Ludii games in a train dataset of ~500 games. For the behavioral characteristic, the authors use a set of game features provided by the game engine on which they have performed PCA (fitted on the train set). For the fitness function, the authors use a set of 3 filters allowing to remove invalid, trivial, severely unbalanced games (among others) and then compute separate scores along several metrics of playablility derived from domain insight and expertise (e.g. amount of time a player has more than 1 option to play, proportion of games that don't end in a draw, etc). The map-elites archive is seeded with 14 games novel to the mutation model.\n\nThe authors report qualitative as well as quantitative evaluations of map-elites on their domain. They demonstrate that the method outperforms (by QD-score) a version where portions of code that have been successfully mutated beforehand are preferentially mutated (GAVEL-UCB). They show that GAVEL is able to create games in novel cells compared with the original Ludii set. They showcase some their created games, selected from the set of promising games by human experts.", "questions": "### Questions\n\n* line 104 did you try not removing these games, does anything bad happen?\n* would CVT-mapelites have worked in this case (to avoid training the PCA on train set games and collapsing variation that doesn't exist in the train set?)\n* Do you have ideas on how to characterize game diversity in other spaces than the BC space? How can we validate that this space adequately capture what humans mean by diverse games?\n* finetuning vs in-context learning: why have you fine-tuned a smaller model instead of asking a large, capable instruction model to perform the task (with examples?) Did you experiment with this? (cost is a valid reason to try using a smaller model, but I am curious on how larger models perform on the task).\n* Open question on fitness:  could the various playability measures be collapsed to a single metric such as learnability, or a computational measure of interestingness, do you have ideas on how this could work or whether it is feasible? (See for instance the interesting but empirically ungrounded work in https://arxiv.org/abs/0812.4360, somewhat related to a recent formulation of open-endeness: https://arxiv.org/abs/2406.04268). This is beyond the current scope of the paper, but I am curious to know if the authors have any thoughts as to how this relates to game domains.\n* Why to you think the UCB variant works less well?\n* The limitation section discusses biasing mutation, do you have any idea on how this could work in practice?\n\n### Suggestions\n\n* (minor) The middle figure of figure 2 could show function expansion;\n* l121 should have a footnote specifying when the training was performed because these things change very fast. (eg codestral can do fill-in-the-middle too now, but was released after the neurips deadline);\n \n### Related work\n\nHere is some work related to the current approach:\n* There is a recent trend for game generation with LLMs, mostly focused on generating geometrical information. The seminal work here is MarioGPT (https://arxiv.org/abs/2302.05981) but there have been several works in this vein since, like https://arxiv.org/abs/2403.12014, https://arxiv.org/abs/2302.05817 or https://arxiv.org/abs/2406.04663\n* On LLM-augmented QD, rainbow-teaming is also an important paper https://arxiv.org/abs/2402.16822", "limitations": "The authors acknowledge both a large set of limitations of their work as well as its broader impact. I would add that an important limitation of the paper is that the proposed implementation of LLM-augmented map-elites lacks generality: fitness, mutation operator and BC all are specific to the Ludii environment.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "* The paper tackles automatic game generation, which is an interesting area of research where modern machine learning methods have much to contribute;\n* The paper is excellently written and very clear (to a reader with knowledge of the area at least). Presentation, pacing and discussion are well executed; it was a pleasure to read. The illustrations are nice;\n* The methodology is very sound, all steps are explained in detail and would be simple to reproduce.\n* The application of evolution through large model (ELM) like methods to game generation seem novel;\n* The cherry-picked examples of games created by the method look engaging, demonstrating the interest of the method;\n* The use of concept vectors are a representative behavioral characteristic (BC) in this domain;\n* The fitness function, while domain-specific, is precise and captures fine-grained notions of enjoyableness of games;\n* Several seeds of all experiments are given, allowing to quantify variability;\n* The authors are very upfront about the limitations of their methods and the claims being made are well supported with evidence.", "weaknesses": "* I worry that the contribution of the paper is limited. While using Map-Elites + LLMs on Ludii is novel, both the fitness function and the BC space used are very specific to games (of the kind presented in the paper), and the paper presents no algorithmic improvement. I would have welcomed methods that work across domains (for instance ask an LLM to self-refine a fitness function and BC space?)\n* While generally, applying a method to a novel domain is interesting in its own right, I worry about the lack of comparative studies in the experiments of the paper. It is good to study map-elites on this new domain but what are the lessons learned? Where are comparisons to other methods, could other generation algorithms (like the ones cited in the related work, or sampling from the model repeatedly without QD, or few-shot prompting a large model with a larger context window) function just as well? Right now this paper reads more like a technical report than a scientific paper, since I do not feel I learned how this complex algorithm works on this domain by careful ablations and comparisons.\n* There could be more information on the study of dynamics of evolution of games in the paper. Why did the authors stop at 500 evolution steps? Was the discovery process saturating? If yes, why? Could more OOD games have been discovered if the process would have been run for longer? If more samples would have been used to seed the archive?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for taking the time to carefully appraise our work. We hope we can adequately address their comments  here.\n\nFirst, while it is the case that our results are specific to games and the Ludii description language, we feel that our approach is general enough to apply to most domain-specific languages. DSLs are used in a variety of contexts and many of them are not well-represented in the training data of LLMs -- motivating a fine-tuning approach. Our fitness function is also general-purpose enough that it would work with minor modifications for other game-generation domains and the broad approach of a heuristic fitness function should in principle be applicable to an even wider range of domains.\n\nNext, we take the reviewer’s point about the lack of comparative studies and attempt to address it through additional baseline experiments. Specifically, we compare GAVEL against directly sampling from our fine-tuned language model and few-shot prompting with GPT-4o. For the direct sampling approach, we randomly select a game from the validation set and mutate it using the same selection and re-sampling process and repeat this process 4500 times (this is to mirror of the 500 generations of GAVEL, since in each generation we randomly select 3 games and perform 3 mutations on each). We then evaluate each of the generated games for fitness and determine the cell they would occupy based on their concepts. This allows us to construct a MAP-Elites archive (i.e. keeping the most fit game in each cell) in order to directly compare against GAVEL. We perform a similar process for GPT-4o, except we provide each of the 14 validation games as part of the prompt and ask the model to create a modification of one randomly-selected validation game. Due to constraints on budget we obtain only 900 samples from GPT-4o and thus compare to GAVEL after 100 generations.\n\nThe results of these baseline experiments are presented in the accompanying PDF, but the short summary is that both baseline methods fall far short of GAVEL in terms of QD score. Pure sampling appears to obtain dramatically fewer high-fitness samples both across the archive and in new cells. While GPT-4o produces a similar number of high-fitness samples in new cells it appears to produce dramatically fewer overall. In both cases, it seems that GAVEL is doing a better job exploring the space of possible games. We will include these results in a camera-ready version of the paper.\n\nFinally, we provide individual responses to the specific questions raised by the reviewer:\n\n- Q1: Did you try not removing the Mancala-style games?\n\n    - A: In initial experiments, our fitness evaluation had difficulty getting reasonable approximations for these Mancala-style games in the limited computational budget we provided. In addition, these games represent a fairly distinct cluster in terms of concepts of ludemes used and we omitted them in order to increase the relative “generality” of the training dataset, though future work could include them.\n\n- Q2: Would CVT-MAP-Elites have worked in this case?\n\n    - A: This is a very reasonable idea, and we did explore CVT-MAP-Elites in an initial experiment as well. Unfortunately running the CVT algorithm over the full concept vector space produced either a very sparse archive (when the number of cells was high) or a relatively collapsed archive (when the number of cells was small), which we attribute to the fact that the tessellation occurs uniformly over the space while the games are not distributed uniformly. \n\n- Q3: Do you have ideas on how to characterize game diversity in other spaces than the BC space?\n\n    - A: Capturing human notions of diversity in gameplay remains an open research question. Prior work does indicate that Ludii concepts capture certain high-level judgements of game genre (see citation [54] in our paper), but outside of this the gold standard would likely remain a human user study. We are in the process of organizing such a user study for future work / a camera-ready version of this paper.\n\n- Q4: Why have you fine-tuned a smaller model instead of asking a large, capable instruction model to perform the task?\n\n    - A: See baseline experiment above\n\n- Q5: Could the various playability measures be collapsed to a single metric such as learnability, or a computational measure of interestingness? \n\n    - A: While this idea is reasonable, we think that the main challenge would be in tractability. Using a measure of learnability as a fitness function would effectively add an inner loop to what is already the main bottleneck of the evolutionary algorithm. Perhaps such an approach would work in domains where generated game code could be compiled to something that runs on the GPU, but it would likely be infeasible for Ludii.\n\n- Q6: Why do you think the UCB variant works less well?\n\n    - A: One hypothesis is that it is very difficult to make any kind of improvement in the fitness landscape GAVEL operates over. The UCB MAP-Elites selector relies on the intuition that cells which produced successful mutations in the past are more likely to do so again. However, if improvement in any given cell is basically equally (un)likely, then the UCB algorithm would effectively waste time re-selecting previously successful cells instead of a more efficient uniform sampling.\n\n- Q7: The limitation section discusses biasing mutation, do you have any idea on how this could work in practice?\n\n    - A: In the context of Ludii, the most likely way to accomplish this would be with a heuristic-based method. It should in principle be possible to write a set of rules that link certain kinds of errors (e.g. unused components in the “equipment” section) with certain program regions (e.g. player actions in the “rules” section). Another possibility is an instruction-based mutation operator to which additional context (e.g. “this game has unused components”) could be provided.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pgh4nlFNcw", "reviewer": "Reviewer_HLJN", "summary": "The paper targets on generating interesting games automatically. To do this, it proposes an evolution-based algorithm, which iteratively mutates the game components using the generalizability of an LLM. The work is based on a previous large-scale game datasets Ludii.", "questions": "1. Is the used CodeLLaMA model the base-version or instructed version? Or which one performs better in your fine-tuning process?\n\n2. The authors mention that the LLM would perfectly reproduce the missing section of code in the process of mutation. The proposed solution is to mutate a set of unseen games. It is surprised to me that the authors do not try to use top-k decoding or noisy decoding, since these methods are more intuitive and easy to me. Or these methods are not good in practice?\n\n3. I am not sure whether this work or the proposed method is greatly reliant on a high-level descriptive game language, or not? If we put this work in broader scenes, e.g. role-play games, do the principles and algorithms in the paper also work?\n\n4. It is not clear how to judge the correctness of the generated code, since the new games are represented in code (if I am wrong please correct me). Solely checking the compilation is not enough to determine if the code is right or not.\n\n\n5. I hope the authors can discuss more recent work in AI generating games, e.g. Instruction-Driven Game Engines on Large Language Models (https://arxiv.org/abs/2404.00276). In this work, the authors automate the development process of poker games based on LLMs.", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. It is an interesting task to me, to automate the process of game design using AI. The authors make a good job in presentation. Some of the details really enlighten me. \n\n2. The authors leverage LLMs to evolve the games iteratively. This is a nice idea, or at least a nice application of today's AI.\n\n3. The word is solid. The experiments are based 1000 different games. While the games are limited in board games, it is still a great job and solid work.", "weaknesses": "1. **Lack of illustrative cases of generated games.** I am interested in how the generated games vary in the process of evaluation (maybe a case in generation 10 and another case in generation 100?). Second, the cases provided in the paper are in Ludii language. It is hard for readers to read immediately. It is better to provide at least a piece of natural language description.\n\n2. **Evaluation is fair.** The evaluation is ok to me, while not novel enough. Most results are predictable and lack of inspiration.\n\n3. **It is still a hard task to evaluate the degree of interest of a game.** The authors propose a number of approaches to do that. Though these approaches are sound to me, most of them have been discussed in previous work. Recent approaches are still far from a faithful and comprehensive evidence for a game being interesting.\n\n4. **Some training and evaluation details are missing.** Please see below.\n\nDespite some limitations, I still believe that this paper offers a solid and interesting work for specific communities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful comments. We acknowledge the reviewer’s points about the difficulty of evaluation and will include a more thorough discussion of these challenges in a camera-ready version.\n\n\nWe also provide individual responses to the specific questions raised:\n\n- Q1: Is the used CodeLLaMA model the base-version or instructed version? Or which one performs better in your fine-tuning process?\n\n    - A: We used the base version since we were fine-tuning on a single task. However, in some of our early experiments, we also fine-tuned instruct models to translate board game instructions from natural language to the Ludii DSL. While we eventually dropped that line of research due to the poor quality of the English descriptions accompanying Ludii games. We didn’t notice any significant difference in performance between the fine-tuned instruct and non-instruct versions of CodeLLaMA.\n\n- Q2: The authors mention that the LLM would perfectly reproduce the missing section of code in the process of mutation. The proposed solution is to mutate a set of unseen games. It is surprised to me that the authors do not try to use top-k decoding or noisy decoding, since these methods are more intuitive and easy to me. Or these methods are not good in practice?\n\n    - A: We agree that diversity-boosting strategies like top-k or noisy decoding are intuitive for increasing the novelty of mutations. However, our early experiments showed that these decoding strategies were ineffective at generating mutations that were both novel and valid. Given our relatively small pool of initial seed games, we found that simply excluding them from the training data was more effective. We will clarify this point in the camera-ready revision of the paper.\n\n- Q3: I am not sure whether this work or the proposed method is greatly reliant on a high-level descriptive game language, or not? If we put this work in broader scenes, e.g. role-play games, do the principles and algorithms in the paper also work?\n\n    - A: The approach we propose is applicable to most formal languages, including DSLs and general-purpose programming languages. The main constraint is that candidate games (or programs) need to be evaluated using a reward function. We achieve this by compiling the game descriptions, playing through them, and using heuristics to quantify the quality of the playthrough. Unlike strategy games, defining similar heuristics for games that feature a stronger human element, such as role-playing games, may be more challenging.\n\n- Q4: It is not clear how to judge the correctness of the generated code, since the new games are represented in code (if I am wrong please correct me). Solely checking the compilation is not enough to determine if the code is right or not.\n\n    - A: Indeed, compilability is only one of the metrics we consider when evaluating the quality or correctness of a game. Instead, we focus on evaluating the quality of playthroughs. We compile sample games into executable programs (they compile to Java bytecode), collect a series of playthroughs using self-play between MCTS agents, and evaluate each game’s playthroughs in aggregate using heuristics that quantify the game’s balance, decisiveness, completion, agency, and coverage. You can find more details about this process on pages 5 and 6.\n\n- Q5: I hope the authors can discuss more recent work in AI generating games, e.g. Instruction-Driven Game Engines on Large Language Models (https://arxiv.org/abs/2404.00276). In this work, the authors automate the development process of poker games based on LLMs.\n\n    - A: We thank the reviewer for pointing us towards this paper. While its approach is quite different from ours - they use LLMs as a game engine to predict next states, while we use them to define rulesets in a DSL - the authors' strong results are exciting. We will consider including it as related work in the camera-ready version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9z5Axzgrqh", "reviewer": "Reviewer_ocXt", "summary": "The paper presents GAVEL (Games via Evolution and Language Models), a system designed for automated game generation. The authors utilize the Ludii game description language (L-GDL) to encode game rules and leverage a combination of evolutionary computation and large language models (LLMs) to generate new and novel games. The system is evaluated both quantitatively and qualitatively, demonstrating the capability to produce playable and interesting games distinct from those in the existing Ludii dataset.", "questions": "- What do the authors imagine could be the impact of the work on the larger ML community\n- Which other application areas outside of games could the system be used for?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The integration of evolutionary algorithms with large language models for game generation is a promising approach, expanding the capabilities beyond traditional rule-based or heuristic methods.\n- The paper provides a thorough evaluation of the generated games, using both automated metrics and expert human analysis, which strengthens the validity of the results.\n- The paper clearly describes the training process of the language model, the evolutionary search algorithm, and the evaluation metrics, allowing for reproducibility and further exploration by other researchers.", "weaknesses": "-  While innovative, the application of this research might be seen as niche within the broader NeurIPS community. The practical impact and scalability of such a system to other domains should be further tested in the future. Additionally, the impact on the broader ML community should be further highlighted.\n- There exists some related work on LLM and created games and level, also including combining LLMs with more open-ended search methods like novelty search, that seems missing, e.g. \n        - Sudhakaran, Shyam, et al. \"Mariogpt: Open-ended text2level generation through large language models.\" Advances in Neural Information Processing Systems 36 (2024).\n        - Todd, Graham, et al. \"Level generation through large language models.\" Proceedings of the 18th International Conference on the Foundations of Digital Games. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful comments and critiques and we hope to address their general concerns in our response. We also appreciate the pointers to additional related work and will include them in a camera-ready version of our paper.\n\nWhile it is the case that the results we present are specific to games and the Ludii description language, we feel that our approach is general enough to apply to most domain-specific languages. DSLs are used in a variety of contexts and many of them are not well-represented in the training data of LLMs -- motivating a fine-tuning approach. Our fitness function is also general-purpose enough that it would work with minor modifications for other game-generation domains and the broad approach of a heuristic fitness function should in principle be applicable to an even wider range of domains. In this way, the broad GAVEL approach could be adapted for effective code synthesis in relatively data-sparse settings.\n\nHowever, even if our method only works effectively for board game generation, this would in our opinion be both significant and general. Most of the world’s population play board games, at least occasionally, and the Ludii dataset contains more than a thousand games from all over the world. In the sense that our model is tasked with producing modifications to an existing game, it has demonstrated a broad and wide-ranging ability to do so. Further, the procedural generation of novel games presents an avenue for future research in open-ended learning.\n\nFinally, we would also like to press the significance of the empirical results of our paper. We are the first to generate not one, but several new and interesting board games. The only directly comparable work is Cameron Browne’s Ludi system from 2009, but that operated in a significantly smaller search space consisting only of connection and line-completion games.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oIc0Hw1B73", "reviewer": "Reviewer_shZT", "summary": "The paper presents a method to generate board games in a domain specific language using an LLM tuned on that language. An LLM is finetuned using fill-in-the-middle training to complete descriptions of board games from a previously gathered dataset. This LLM is used as a mutation operator in a quality-diversity algorithm. Evaluations start generation from a held-out set of seed games and show the algorithm produces more variation of high quality than an ablation. Preliminary expert human evaluations are favorable for the quality of the newly generated games.", "questions": "- Section 3.1\n\t- How many games were filtered by each step of the process? It looks like ~1/2 the games were filtered (from 1182 to 574+14).\n- Section 4.2\n\t- How is the concept vector defined for new games? Is it derived from the seed game concept vector (potentially mutated)?\n- (minor) lines 225-227: \"In cases like ours where fitness values can be negative, each fitness value is incremented by the minimum possible fitness (i.e. -2) before being summed to ensure that the QD score increases monotonically over time.\"\n\t- Perhaps this is a confusion on my part: if the minimum is fixed and samples are only added to the archive when increasing in fitness, this (seems to) imply that the score can only increase. Offsetting by the minimum may shift the minimum value obtained to be positive, but should not change the process of monotonic improvement.", "limitations": "Yes. The paper describes the limitations of the current approach and provides potential extensions to mitigate these weaknesses. This includes generating unused code, limited diversity, construction of the archive, and automating human evaluation.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "# originality\nLow\n- Using LLMs as mutation operators was previously established as a methodology (as referenced).\n- Fine-tuning LLMs to produce domain-specific code is also well-explored.\n- The primary novelty is the target dataset used.\n\n# quality\nReasonable\n- Great to have statistical testing of differences!\n- The new domain makes it hard to have alternative baselines, but this makes it difficult to know how successful the algorithm is at solving the base domain task. The quality-diversity metric is \"internal\" to the algorithm in the sense of being a proxy metric to produce the desired \"good games.\" The preliminary expert evaluations are promising, but not a strong piece of evidence (yet).\n\n\n# clarity\nGood\n- Clearly articulates the problem and approach. Introduces domain-specific information to help readers understand the DSL and its features.\n- The paper is direct about limitations and potential extensions.\n\n\n# significance\nModest\n- Of interest to the game generation community.\n- Potentially of some interest to the code generation community, but the techniques are not particularly novel.\n- The expert human evaluation of potentially novel games is a positive step toward impact.", "weaknesses": "Fundamentally the paper rests on the success of the algorithm at generating games (given the relatively established techniques being employed). This is hard to gauge from the experiments so far: the QD metric does not directly measure success against a ground truth and the comparison to GAVEL-UCB is an ablation (thus relative change) without baseline algorithms to compare against.\n\nHow could the paper provide more persuasive evidence that GAVEL is advancing the problem of board game generation?\nPerhaps consider comparing to few-shot prompted code LLMs as a more \"naive\" baseline. How does GAVEL compare in terms of number of tokens needed to produce games and the quality of the best of those games?\nOr perhaps there is a simple ablation that replaces MAP-Elites with a reasonable alternative and shows that GAVEL produces more diverse and/or better games when consuming the same number of steps of generation (or raw tokens produced through LLM inference). These experiments (unfortunately perhaps too much for the discussion period) would make it clearer that the more direct ways to solve this problem are not sufficient. Matching on inference (token) budget would further provide evidence of better efficiency. Or perhaps this would show GAVEL can asymptote to greater quality in final artifacts even if it demands a larger inference budget.\n\nFor the human evaluation, is there recorded feedback or other narrative information from responses that might strengthen the claims that GAVEL is succeeding?\n\nMy core concern is the techniques used are not particularly novel in isolation and in combination, but the domain is too novel to know if the algorithm is succeeding from the experiments so far. I would be persuaded to increase my score if I saw stronger results that GAVEL is truly solving game generation, or is a superior approach to reasonable alternatives.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful comments and critiques. We hope to address their primary concern through additional baseline experiments that were performed during the response period.\n\nSpecifically, we compare GAVEL against directly sampling from our fine-tuned language model and few-shot prompting with GPT-4o. For the direct sampling approach, we randomly select a game from the validation set and mutate it using the same selection and re-sampling process and repeat this process 4500 times (this is to mirror of the 500 generations of GAVEL, since in each generation we randomly select 3 games and perform 3 mutations on each). We then evaluate each of the generated games for fitness and determine the cell they would occupy based on their concepts. This allows us to construct a MAP-Elites archive (i.e. by  keeping the most fit game in each cell) for the baseline in order to directly compare against GAVEL. We perform a similar process for GPT-4o, except we provide each of the 14 validation games as part of the prompt and ask the model to create a modification of one randomly-selected validation game. Due to constraints on budget we obtain only 900 samples from GPT-4o and thus compare to GAVEL after 100 generations. In a camera-ready version of the paper we will extend the GPT-4o baseline to the full 4500 samples and experiment with including additional context in the prompt.\n\nThe results of these baseline experiments are presented in the accompanying PDF, but the short summary is that both baseline methods fall far short of GAVEL in terms of QD score. Pure sampling appears to obtain dramatically fewer high-fitness samples both across the archive and in new cells. While GPT-4o produces a similar number of high-fitness samples in new cells it appears to produce dramatically fewer high-fitness samples overall. In both cases, it seems that GAVEL is doing a better job exploring the space of possible games. We will include these results in a camera-ready version of the paper.\n\n\nIn addition, we provide individual responses to the specific questions raised:\n\n- Q1: How many games were filtered by each step of the process?\n\n    - A: From the 1182 games in the full dataset, we first filter down to 1033 games in the “board” category. Removing the Mancala-style games brings the dataset to 825 games, and filtering by tokenized length brings us to our final 574 training games and 14 validation games.\n\n- Q2: How is the concept vector defined for new games?\n\n    - A: Ludii can automatically compute the concept vector for any game that successfully compiles: each feature is derived from the combination of “ludemes” (essentially, keywords) that are used in the game description.\n\n- Q3: Doesn’t the QD score increase monotonically even without the offsetting of individual fitness values?\n\n    - A: In most MAP-Elites implementations (and in ours), a sample can get added to a previously-empty archive cell regardless of its fitness score. This means that, in our case, games in novel cells with negative fitness scores can cause the QD score to decrease. We will make this distinction more clear in the camera-ready version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank each of the reviewers for taking the time to consider our work. In the attached PDF we include the results of additional baseline experiments performed during the response period that compare GAVEL to a pure sampling approach and few-shot prompting with GPT-4o. The details and take-aways of these experiments are included in the table captions.

Meta-review: An innovative paper, rated highly for clarity, on the very timely topic of generating many novel and diverse games using code-LLMs. The reviewers are overall positive, and I too see a large potential for this work, both because generating interesting games is valuable in itself, and because I can see the method's components transfer to related domains of generating tasks or code. I'm looking forward to the major revisions in camera-ready with the additional baselines, the user-study, and an appendix that documents (or revisits) all these mentioned "initial experiments" mentioned in the author feedback.
Decision: Accept (poster)