Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Games
Authors: Dongmin Park
Minkyu Kim
Beongjun Choi
Junhyuck Kim
Keon Lee
Jonghyun Lee
Inkyu Park
Byeong-Uk Lee
Jaeyoung Hwang
Jaewoo Ahn
Ameya Sunil Mahabaleshwarkar
Bilal Kartal
Pritam Biswas
Yoshi Suhara
Kangwook Lee
Jaewoong Cho
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
Agents
Benchmark
Games
Primary area: datasets and benchmarks
TL;DR: We introduce a comprehensive benchmark for training and evaluating LLM agents on diverse real-world video games

Abstract: Large Language Model (LLM) agents are reshaping the game industry, by enabling more intelligent and human-preferable characters. Yet, current game benchmarks fall short of practical needs: they lack evaluations of diverse LLM capabilities across various game genres, studies of agentic modules crucial for complex gameplay, and fine-tuning datasets to adapt pre-trained LLMs into gaming agents. To fill these gaps, we present Orak, a benchmark for training and evaluating LLM agents across 12 popular video games spanning all major genres. Using a plug-and-play interface built on Model Context Protocol (MCP), Orak supports systematic and reproducible studies of agentic modules in varied game scenarios. We further release a fine-tuning dataset of expert LLM gameplay trajectories spanning multiple genres, turning general LLMs into effective game agents. Orak offers a comprehensive evaluation framework, including game leaderboards, LLM battle arenas, and in-depth analyses of input modality, agentic strategies, and fine-tuning effects, establishing a foundation towards versatile gaming agents. Code and datasets are available at https://github.com/krafton-ai/Orak and https://huggingface.co/datasets/KRAFTON/Orak.

Published as a conference paper at ICLR 2026
ORAK: A FOUNDATIONAL BENCHMARK FOR TRAINING AND EVALUATING LLM AGENTS ON DIVERSE VIDEO GAMES
Dongmin Minkyy Kim 1 Beongjun ; Kim"t, Keon Lee't, Jonghyun Lee"t, Inkyu Park't 1ark -Byeong-Uk Lee" Jaeyoung Hwang" 1 Jaewoo Ahn :21, Ameya S. Mahahaleshwarkar", Bilal Kartal,, Pritam Biswas", Yoshi Suhara", Kangwook Lee Jaewoong Cho KRAFTON, Seoul National University, "VIIIA, "University of Aacconiin-Madioon 'Ludo Robotics
ABSTRACT
Large Language Model (LLM) agents are reshaping the game industry, by en- abling more intelligent and human-preferable characters. Yet, current game benchmarks fall short of practical needs: they lack evaluations of diverse I.LM capabilities across various game genres, studies of agentic modules crucial for complex gameplay, and fine-tuning datasets to adapt pre-trained LLMs into gam- ing agents. To fill these gaps, we present Orak, benchmark for training and evaluating LLM agents across 12 popular video games spanning all major gen- res. Using a plug-and-play interface built on Model Context Protocol (MCP), Orak supports systematic and reproducible studies of agentic modules in varied game scenarios. We further release a fine-tuning dataset of expert L.LM game- play trajectories covering multiple genres, turning general LLMs into effective game agents. Orak offers a united evaluation framework, including game leader- boards, LLM battle arenas, and ablation studies of input modality, agentic strate- gies, and fine-tuning effects, establishing a foundation towards versatile gaming agents. Code and datasets are available https://github... http:://ittubb.com/aftooo
www.oootttooaiiOOooooo and
INTRODUCTION
Large Language Model (LLM) agents are revolutionizing various industries (Wang et al., 2024a), and well-established benchmarks play a key role in unlocking their abilities on complex tasks, e.g., Coding (Hendrycks et al., 2021: Zhuo et al., 2024), Web Search (Liu et al., 2023; Pan et al., 2024), and Scientific Research (Mühlbacher et al., 2024; Zhang et al., 2025). In the game industry, there growing interest in using LLM agents to enhance user game experiences, e.g., more intelligent non-player characters (NPCs), monsters, or companions (NVIDIA, 2025). Also, games that simulate dynamic and uncertain real-world scenarios can serve as effective test-beds for evaluating agents' high-level decision- i.e., system 2 reasoning (Li et al., 2025). In response, many benchmarks have been proposed to assess the ability of LLMs by playing games (Hu et al., 2024b; Ahn et al., 2025).
While these benchmarks have effectively utilized games to evaluate LLMs' general capabilities, they exhibit three major limitations: 1) they often rely text-only games 2D-grid simulators rather than complex real video games, 2) they offer insufficient assessment of agentic modules, such as self-reflection, memory, and tool use, which are essential complex gameplay, and 3) they lack fine-tuning datasets necessary to adapt pre-trained L.MMs into effective gameplay agents, which significantly hinder the adoption of LLM agents in real-world video games.
To this end, we present Orak, a foundational benchmark designed to evaluate LLM agents across diverse video games. As shown in Figure 1, Orak includes 12 video games played by millions to billions of users worldwide: Street Fighter III, Super Mario, Ace Attorney, Her Story. Pokémon Red, Darkest Dungeon, Minecraft, Stardew Valley, StarCraft II, Slay the Spire, Baba Is You, and 2048.
Equul contribution. Cooee contribution.

---

Published as a conference paper at ICLR 2026
Environment with Video Games
Orak
Athredtre
Stattey
.ddls
Modes
Ele
GPT Go
38.9
GPT 46
1622.9
A benchmark toward geming LLM agent
Genzai
1053.0
2045
Gameplay saderbourd
LLM Arema Arena
LLM A Agent
MCP mierfac
A Strotngg
Evaluator
AGENT
Adventom
Keflecton Agen
Tools
Confgs Cane
Reborn Soom
Pizzle
Planzing Agent
-II.M
11Mss
Agenti Modkdas
Agast
Stroleyy
A Study
Fine- Tinettninn Study
Figure I: Overview of Orak, a benchmark designed train and evaluate LLM agents across 12 video games across genres. Using MCP as a plug-and-play interface, it ensures systematic assessment, supporting gameplay leaderboards, battle arenas, and studies on agentic modules and fine-tuning.
These games span 6 major game genres, i.e., action, adventure, rol-ppayinng simulation, strategy, and puzzle, enabling a comprehensive assessment of key abilities required for versatile gameplay; action games enable testing fine-grained player control, adventure games challenge long-term memory and error handling, and strategy/puzzle games require complex logical reasoning and multi-step planning. With the use of real video games, Orak also ensures evaluation rich, dynamic environments with varying stages, levels, and story-driven quests, which are even challenging for humans.
To enable consistent evaluation of rapidly evolving LLMs, introduce a plug- lllug-an--oi and-play interface using Model Context Protocol (MCP) (Hou et al., 2025), allowing LLMs to seamlessly interact with agentic modules in gameplay. Each game environment and agentic module package operates as an independent MCP server, providing game mechanics (e.g., retrieving game states, executing game steps) or agentic strategies (e.g., reflection, planning) as callable tools LLMs. During evaluation, LLM interacts with these servers by sequentially retrieving game states, performing action inference using agentic modules, and executing game steps, which enables streamlined evaluation across diverse games and supports controlled studies of various agentic modules.
In addition, we release fine-tuning dataset that aims to transform pre-trained LLMs into effective gaming agents. The dataset consists of game interaction trajectories generated by expert LLMs, e.g., GPT-4o, using core agentic strategies on all games in Orak. These trajectories encapsulate koowlegn on how and when to use the agentic strategies to play various genres of games, leading to more resource-efficient and effective gaming agents.
Our benchmark offers comprehensive evaluation dimensions, including game score leaderboards, competitive I.LM battle arenas, and ablation studies of visual input state, agentic strategies, and fine-tuning effects. Extensive experiments Orak with 15 LLMs reveal that (1) proprietary LLMs achieve superior performance across games with significant gaps from open-source LLMs, (2) their performance gap becomes narrow in battle scenarios, (3) proprietary LLMs benefit from extended agentic workflows, while open-source LLMs show limited gains, (4) models fail to extract sufficient value from visual inputs yet, and (5) fine-tuning enables effective transfer of gameplay meta-knowledge from larger LLMs to smaller ones, leading to generalization in intra-game, out-of- distribution (OOD) game, and non-game unseen scenarios like math and web interaction. We believe that Orak not only establishes a foundation for building gaming agents but also serves as critical benchmark for evaluating general LLMs on realistic, long-horizon decision- makinn tasks.
2 RELATED WORK
Playing Games by LLMs. Many works have explored the use of L.LMs for gameplay. Early efforts focused on text-based games such as Jericho (Hausknecht et al., 2020), Zork (Tsai et al., 2023), and TextCraft (Prasad et al., 2023), where LLMS navigate textual environments with reasoning. Subse- quent work shifted toward 2D-grid games, including Chess (Feng et al., 2023), NetHack (Küttler et al., 2020), and Crafter (Hafner, 2021), where spatial reasoning and puzzle-solving skills became more important. More recently, several studies have applied LLMs, combined with agentic work- flows, to play more complex video games, such as Minecraft (Fan et al., 2022: Wang et al., 2023), Civilization (Qi et al., 2024), Pokémon (Hu et al., 2024c), and StarCraft (Ma et al., 2024). However,
2

---

Published as a conference paper at ICLR 2026
Benchmarks
Game Domain
Full Genre
# Games
Model Type
Ageot Ablation
Fine-tuning Set
GAMA-hench (Huang al., 2024)
Text
X
LIM
GaneBench (Costarelli el al., 2024)
Text
K
LLM
X
GameArena
2024a)
Text
K
LIM
SmartPaa (Wu et al., 2023)
Text/2D-grid
K
LLM
X
Balrog (Paglieri et al., 2024)
Texid2Dgrid
X
LLM/VLM
X
LVL.M-Playrruund (Wang al., 2025a)
2D-grid
K
VLM
X
Cradle (Tan et al., 2024)
Video
X
VLM
X
V-MAGE (Zheng al., 2025)
Video
VLM
DSGBench (Tang ot al., 2025)
Video
X
LLM
X
I.Gamme-FB Rench (Hu et al., 2025)
Video
X
LLMIVIM
Orak (Ours)
Video
LLM/VLM
Table Game Benchmark Comparison. 'Full Genre' means whether six major genres are fully cov- ered. 'Model Type' indicates whether the benchmark supports LLMs VLMs. Unlike prior bench- marks, Orak the only benchmark that fully covers all major genres, supports both LLMs/VLMs, provides ablation studies for agent modules, and releases fine-tuning set.
these approaches rely on manually customized agentic workflows for each specific game, limiting their usability toward developing general gaming agent.
Evaluation Benchmarks for LLMs with Games. As gameplay requires complex cognitive abili- ties, e.g., context understanding, logical reasoning, and error handling, several recent benchmarks have sought to evaluate LLMs or Vision Language Models (VLMs) games (Hu et al., 2024b). GAMA-Bench (Huang et al., 2024), GameBench (Costarelli et al., 2024), GameArena (Hu et al., 2024a), and SmartPlay (Wu et al., 2023) focus on text-based games, assessing LLMS' ability navi- gate and reason on textual environments. Barlog (Paglieri et al., 2024) and LVLM-Playground (Wang al., 2025a) are mainly based on 2D-grid games, such as TicTacTTe and Chess, evaluate the spatial and visual reasoning capabilities of LLMs/VLMs. Using video games, Cradle (Tan et al., 2024) evaluates VLMs in adventure and simulation games, V-MAGE (Zheng al., 2025) assesses VLMs on 5 action games, DSGBench (Tang et al., 2025) validates LLMs on 6 strategic games, and AMGame-Bene's (Hu et al., 2025) mainly focus on puzzle games. Despite their contributions, existing benchmarks lack full coverage of game genres, omit in-depth studies on agentic modules, often rely on visual inputs, and under-explore the way to align pre- vre-trained LLMs into versatile game agents. Table 1 summarizes the key characteristics of game benchmarks.
LLMs toward Agents. To enhance agent capability, many efforts have proposed agen- tic strategies, e.g., chain-of-thougii reasoning (Wei et al., 2022; Yao et al., 2023), self-reflection (Shinn et al., 2023; Park al., 2023), hierarchical task planning (Huang et al., 2022b;a), and skill li- braries (Wang et al., 2023). Complementing these advancements, efforts have focused on fine-tuning strategies tailored LLM agents, which span two main directions: data-centric approaches, which involve fine-tuning on curated expert demonstrations (Chen et al., 2023; Zeng et al., 2023; Chen et al., 2024); and framework-oriented approaches, which focus on learning from agentic interactions (Feng et al., 2024; Chen et al., 2025; Putta et al., 2024). Notably, FireAct (Chen et al., 2023) emphasizes unified data formatting and CodeAct (Wang et al., 2024b) highlights the importance of high-quality curation of training trajectories. However, fine-tuning methods for game playing agents remain largely explored. Unlike structured tasks in web, programming, or math domains, games involve large, dynamic, and partially observable state spaces, which requires agents generalize across variety situations and learn diverse behavior patterns, posing unique challenges.
3 ORAK
We propose Orak, a benchmark designed to evaluate LLM agents across diverse video games. Figure 2 illustrates the evaluation pipeline of Orak, which is self-explanatory with code structures. By integrating the MCP interface with game environments and agentic modules, Orak enables systematic and plug-and-play evaluation of backbone LLMs with agentic strategies across various games. For evaluation, the game score is obtained by simply configuring the game, LLM backbone, and agentic strategy in eval.po py. At each game step, the game observation is retrieved, the specified agent strategy is executed by the LLM, and the resulting action is applied to the game. This loop continues until the game ends or reaches the maximum step limit, after which the game score recorded. Note that, with MCP interface, users can readily customize their agentic strategy, i.e., calling single
3

---

Published as a conference paper at ICLR 2026
eval.py
Met intattom de durttiaaaiinn Matpappoopo
env.py
Class (valustorcr()):
Ganelinv env):
ecp_env no0_Ganefnvictg-gom www.eeccggggggg....ooo
def obs2text(obs): neturn text def testZact(text)) return act def step(art): return obs
ecp agent PCPPAgent()
100d 1in(cff. httpc/ff.....aaaam nanc)
play):
step (MM_ee)x_stepp))
Input nep_ pop_em..call ("cbutitt"" obs)
agent.py
Touls
llm.py
Agentic Strategy
dlactiin watteaoaallin Merups
Class Agent():
Menuy
def lead [i "got":
Lext scp, agent (cl. 11a)
MESP insettrr
raeory Meeory()
Knowledgn
Skill moniger
return (Pibase()
* Clando
stttpp,,, lect)
reflection(lie, text)
_ourppt
Prompts
neturn ClaudeBase()
"claude"
score ecp www.caallllll........ ("evaluate")
def planning( Lext)
Planning
return taxt ouzgut
elses
return LocalBase()
Figure 2: Evaluation pipeline of Orak. Game scores are computed via eval. py by simply con- figuring game, LLM backbone, and agentic strategy. Orak supports two types of submissions: (I) customizing 11m. 1.mmpe py with new backbone LLMs, and (2) customizing agent py with new agentic strategies. The agentic strategies callable by LLMs via MCP interface in eval py (in grey box).
Required LLM Capabilities
Special Ressoning
Taricaal
Reaisohing
Lopletext
Uddrstuaddng
TEl
.661
OE
OHE
2005
(6) Street Fighter
(6) Super Manio
(c) Ace Atoomay
(d) Her Story
(c) Pokeémn Red
(9) Dorkest Uungron
Rule
Following
uru
Long-tchi
Playning
Oddds
Handligg
Hmddling
Mineeraftt
Standew Valley
(i
() Slay the Spire
Baba
Figure 3: LLM capabilities required to play games Orak. The color theme (red, yellow, etc) represents game genres. See Appendix A for genre categorization details.
agentic module or multiple agentic modules sequentially. In the following section, we describe the 12 game environments in Orak, along with the submission guidelines for the benchmark leaderboard.
3.1 GAME ENVIRONMENTS
LLM Capabilities Required. LLM agents require diverse capabilities to play games. Figure 3 summarizes the capability levels needed for each game in Orak, measured using principled criteria adopted from the context of game design (Wu et al., 2023; Koster, 2013):
Rule Following (RF). The level is measured by the extent to which adherence to rules is required for gameplay (1: single rule, 2: fewer than 5 rules, 3: 5 or more rules).
Logical Reasoning (LR). The number of LLM's reasoning hops required to determine an in in-game
action (1: 1 hop. 2: to 3 hops, 3: 3 or more hops).
Spatial Reasoning (SR). The level of spatial understanding required for gameplay (1: not necessary, gameplay).
2: required in specific situations, 3: critical
Long-text Understanding (LTU). The extent long-context comprehension required for gameplay
(1: few lines, 2: few paragraphs, 3: longer than one page with 500+ words).
Long-term Planning (LP). The extent to which strategic planning is required (1: not necessary, 2:
planning for up to 3 sequential actions, essential plan more than sequential actions).
Error Handling (EH). The extent to which error correction is required during gameplay (1: not necessary, 2: requires one-step rollback, 3: requires multi-step rollback and re-planning).
Odds Handling (OH). The extent to which understanding randomness is required for gameplay (1: not necessary, 2: randomness exists in game, 3: randomness is critical to core gameplay).
The capability level is measured on scale of to by 8 human participants, and the moderate value is reported. Since most video games are designed to require various cognitive abilities even for humans, they tend to require high levels of LLM capabilities in many dimensions. For example, action games, Street Fighter III and Super Mario red, require spatial reasoning and rule following, more than long-con context understanding and planning, while adventure games, Ace Attorney and Her Story in yellow, emphasize long-text understanding and logical reasoning due to the need to comprehend long storylines. More detailed analysis of required L.M capabilities for each game is in Appendix B.
4

---

Published as a conference paper at ICLR 2026
Game Description. For each game, provide a brief description of (1) the game state, (2) the action space given to LLMs, and (3) the evaluation task and metric. More detailed explanations of each environment are elaborated in Appendices C-N.
(a) Street Fighter III (Capcom, 1997) is a 2D side-scrolling fighting action game with 20 unique characters, each with distinctive skills. (1) Game state: The player character, opponent character, remaining time, each player's ' health, super- super-bar, bar, stun-bar gauges, and the distance between the two characters. (2) Action space: 15-20 discrete actions: 'move closer', 'low punch', 'high kick', etc. (3) Evaluation task: Beating the game bot; performance is measured by the number of stages cleared.
(b) Super Mario (Christian Kauten, 2018) is a side-scrolling game where the player controls Mario to avoid obstacles, defeat enemies, and reach the flag. (1) Game state: The positions (x.y) and sizes of obstacles and enemies extracted from the current game state. (2) Action space: Mario keeps moving to the right, and LLM decides the jump level, discretized in 6 bins. (3) Evaluation lask: Reaching out to the final flag: performance is measured by the horizontal distance that Mario travels before dying.
(c) Ace Attorney (Capcom, 2001) is courtroom adventure game where players act as defense attornys, gathering evidence and witnesses to prove their client's 's innocence. (1) Game state: Dialogue history, collected evidence, court records profiles, etc. (2) Action space: Player's 's courtroom actions: advancing dialogue, accessing court records, pressing witnesses, and presenting evidence. (3) Evaluation task: Performance is measured by response correctness and total steps taken.
(d) Her Story (Barlow, 2015) is an interactive adventure game where players explore police interview clips to uncover hidden truth. (1) Game state: History of queries and search results with metadata for the first 5 clips (visual description, date, viewing status, and transcript if played). (2) Action space: Searching for clips with keywords, or selecting a video to play. (3) Evaluation task: Uncover the truth; Performance is measured by the number of distinct video clips viewed to complete the game.
(e) Pokémon Red (Game Freak, 1996) is a role-playing game where player explores, collects Pokémon, and battles other trainers to progress the storyline. (1) Game state: Player's 's location, party Pokémon (species, level, HP, status), inventory, battle state, and screen text. (2) Action space: Choosing level tools low-level joypad actions. (3) Evaluation task: Defeat Brock, the first gym leader; Progress measured by how many of 12 predefined storyline flags are triggered.
(f) Darkest Dungeon (Red Hook Studios, 2016) is a role-playing game where heroes explore dungeons while managing stress and resources. (1) Game state: Party status (character stats, health, stress, and status effects), available skills, and enemy encounters. (2) Action space: Combat actions like 'attack', 'heal', and 'swap'. (3) Evaluation task: Complete the first expedition; Performance is measured by the sum of the successful combats, the survived heroes, and their remaining stress.
(g) Minecraft (Mojang Studios, 2011) 1) is an open-ended sandbox game where players explore world, gather resources, and survive by placing and breaking blocks. (1) Game state: The player's 'ss position, inventory, health status, the nearby blocks and biome, etc. (2) Action spuce: Executable JavaScript code within the Mineflayer environment (contributors, 2013). (3) Evaluation task: Crafting a target item; performance is measured by whether the item is collected in the inventory.
(h) Stardew Valley (ConcernedApe, 2016) is an open-ended life simulation game where players farm, fish, mine, and explore. (1) Game state: Player's location, energy, inventory, crop status, soil status, date, and weather. (2) Action spuce: Leaving the house, entering the house, sleeping, buying seeds, tilling soil, watering, harvesting, and selling crops. (3) Evaluation task: Earning the most money by harvesting crops within the first 13 in in-game days; performance is measured by total profit.
(i) StarCraft II (Blizzard Entertainment, 2010) is a real-time strategy game where players gather resources, construct buildings, train units, and command armies to defeat opponents. (1) Game state: Resource levels, unit/building counts, production queues, research progress, and observed enemy info. (2) Action space: 72 discrete actions, including unit training, building, research, and strategic operations. (3) Evaluation task: Beating built-in Al bots; performance is measured by the win rate.
(j) Slay the Spire (MegaCrit, 2017) is deck-building roguelike game where players ascend multi-floor tower, battling enemies and building decks. (1) Game state: Player's 's class, deck, hand, health, relics, energy, enemies" intents and statuses, and current floor. (2) Action space: Playing card during combat, ending the turn, and selecting a card reward after combat. (3) Evaluation task: Defeating the final boss at the top floor; performance measured by the number of floors reached.
(k) Baba Is You (Hempuli, 2019) is puzzle game where a player manipulates the rules by moving word tiles on board. (I) Game state: Coordinates of text and object tiles, and active rules. (2)
5

---

Published as a conference paper at ICLR 2026
Action space: A single movement 'up', 'down', 'left', and 'right', sequence of such moves. (3) Evaluation task: Solving the first stage; if the stage is not cleared, partial credit is awarded based on sub- -oobggaas (e.g., breaking the 'Wall Is Stop' rule).
(I) 2048 (Cirulli, 2014) is a sliding tile puzzle game that aims to combine numbered tiles on a 4 x4 grid board create tile of the value 2048. (1) Game state: The current configuration of the 4x4 grid, where each cell contains either a number (power of 2) or is empty. (2) Action space: Four discrete actions; "up'. 'down', 'left', and 'right'. (3) Evaluation task: Creating the 2048 tile; performance is measured by the normalized progress toward creating the 2048 tile.
3.2 SUBMISSION GUIDELINE TO BENCHMARK LEADERBOARD
Models. Participants can submit pre-trained LLMs, VLMs, their fine-tuned versions. Unless otherwise specified, all models will be evaluated under the default agentic strategies for each game specified in Section 5.1.
Agentic Strategies. Participants can also submit new agentic modules and strategies. All submitted strategies should follow a consistent structure across games, e.g., by maintaining fixed sequence such as reasoning, planning, and using specific tool.
4 FINE-TUNING: ALIGNING PRE-TRAINED LLMS INTO GAME AGENTS
Data example from Reflection module
Data example from Action module
Analyze Mario's action using difference and provide critiques for improving his action.
(Retrieve the self-reflection from memory)
Analyze the current game state, and decide the best action. should only respond
X
You should only respond in the format below:
format below:
# # Self-reflection
# # Action
(Describe self-reflection here)
Jump Level: a (where an integer from 6)
# # Past Game State
# # Game Stare Mario (100,100) Position of all objects:
Mario at (100,100), Bricks (120, 100), (120, 150)
S
S
# ### Current Game State
Mario at (100,100), Bricks (120, 100), (120, 150)
Bricks at (120, 100), (120, 150)...
# Self-reflection
y.s
# # Action
Mario blocked by bricks. Jump higher get past.
Jump Level: 6.
Table Fine-tuning data examples when playing Supermario with 'reflection' agent.
We collect the fine-tuning dataset from expert LLMs, e.g., GPT-4o and 03-mini, playing all 12 games in Orak using several agentic modules. This dataset with environment interaction trajectories encapsulates meta-knowledge on how to use the agentic strategies to solve diverse game genres.
Data Format. We denote LLMs' gameplay trajectory as T == (TI...., TT where T is the number of game steps, and T. denotes the sequence LLM inferences executed by agentic strategies at game step t. Each LLM inference sequence is represented as T {(X", S, Y.:)) ) where a, E ('reflection', 'planning'. ..... 'action') is the i-th agentic module sequence, X" is the prompt for agentic module S is the game state, and y is the corresponding response of LLM. Table shows detailed data examples Tf
Data Selection. For each game in Orak, we collect gameplay trajectories of expert LLMs, until we have more than 1000 LLM inference sequence T in all T collected. To ensure high-quality data, we sort the collected T in terms of the game score and select the trajectories with the highest game scores until the number of selected 7 exceeds 300. All selected trajectories follow the reflection-planmil action' sequence, so that we have around 300 samples for each agent module. By performing data selection on all 12 games, our fine-tuning set consists of approximately 11k samples.
Data Augmentation. To enhance the linguistic diversity, we augment each data sample T by paraphrasing. We prompt GPT-40 to rephrase the game prompt X while preserving all game-related information, generating 10 augmented samples for each sample т. See Appendix O for prompting details and the effect of the number of augmentations.
Our fine-tuning dataset is mainly for supervised fine-tuning (SFT). While dynamic data extraction from the environment could enable reinforcement learning fine-tuning, we leave this for future work.

---

Published as a conference paper at ICLR 2026
Come
Adventure
Simulation
Sinnegy
Puzsle
Cames
SP3
SaperMario
AceAttomey
HerSior
Pokééoo
DarkeaD)
Minccrafi
Stanlew
StarCmi22
SlaySyire
BubufsYoa
2046
Rank
3.2-1-32--1
133022
0,0aaa
0dless
6.am
13.5
3.233-
200
0.3
Ques
20.0.0
13.4
20.0..44
0.0.00
0.0
0.00a
0.0
11.3
0.1:11
12.1
0-2s-255-7
16.2nm
27.2
9.3
3.5
0.0ce
88.8ata
0.0aaa
0.0ga
5.lees
200
0.ess
10.5
Mintion-4B
16.7
35.7.1
4.G
0.0ce
0.0aaa
033eas
200
011s
11.6
Minimon88B
23.
3133
29.9
8.2
0.0zm
63
0.0
0.0ga
0.0ga
0.0
20.
0.72x
10.1
LJam23,3 70B
23
32,0na
53.9.214
16.4
16.7
87.2
0,0
409
0.0
3,0
200
1.4
1.3
Qwen2.5-728
2677
20.9ga
14.9
41.0gah
27.8
54.0gr
0.0gas
180
0.0g
9.0gs
200gr
0.2g11
8.5
GPT 40 mini
16.7
28.8
28.4:1a
21.2
0.0.co
81.3
46.0.ca
75.0
3,3
13:3
1.1
9.2
GPT-4a
29.7a:45
34.1
85.3g1
64.2
38 38.9
93
71.0gza
100,0
23.6
200
5.6=
3.6
GPTS
19.52:1
31,61m
59
1:315
74,9
88.9..:
92.9um
73.0us
92.3.s
25.0uma
26.2.1
100.0.co
10.2
3,6
03-mini
33,3.33
34,9
91.7-15
66.5
0.0.ca
39.0..
75.0
55.1-s
25.0
15,0kaa
73.3-
25,3
4,0
Gemini 2.5 pro
13.3
38.0
53.7
67,5
K3.3-4
41.7.t
75,0
592.mu
100.0
$1.9 mi
73.3-42
5.1.21
3.5
Claalo-3.7
16.7.11
31.7ac
81.9ai
62.9
63.9
89.0
75.0
53.600
50.
15.0
467==
5.3
5.1
Deepoeck R1
20.0.00
28.7a0a
83.3
67.2
75.0.m0
91.7am
41.7a0a
66.1.n
50.0
24.9
20.0cc
11.5
5.1
Human (Novice)
20.0.300
100,0.0
87.8.s
72
86.1
90.3
70.8.09
69...9
33.3
$2.9
1000
22.7
Table 3: Performance of LLMs Orak with default agentic strategies. The best scores for each
game are highlighted in bold, and the average LLM rankings across all games are reported.
EXPERIMENT
5.1 EXPERIMENT SETUP
Models. We validate the performance of 15 LLMs using states provided in text format. The models include open-source LLMs: LLaMA-3.2-118 (Grattafiori et al., 2024), LLaMA-3.3- 72B (Grattafiori et al., 2024), Qwwn-225-3B/77B/72B (Yang al., 2024), and Minitron-4B/SB (Sreeni- vas et al., 2024), and 7 proprictary LLMs: GPT-5/4o/4o-mini (Achiam et al., 2023), 03-mini, Gemini-2.5-pro (Team et al., 2023), Claude-3.7 (Anthropic, 2025), and DeepSeek-RI (Guo et al., 2025). In addition, we study the effects of incorporating image inputs on 5 multi-modal LLMs: Qwen2.5-x1-7B/32B (Bai et al., 2025), GPT-4o, Gemini-2.5-pro. and Saaude-..7-oomme
Default Agentic Strategies. For each game in Orak, we select the most effective agentic strategy over GPT-4o as the default agent strategy. Specifically, For Street Fighter III. HerStory, Darkest Dungeon, and 2048, we use 'zero-shot' action inference agent. For Super Mario, Pokémon Red, Stardew Valley, StarCraft II, Slay the Spire, and Baba Is You, we use a 'reflection that sequentially performs self-reftection, subtask planning, and action inference, integrated with memory at each game step. For AceAltorney, we use 'reflection' agent. For Minecraft, we use a maiil-mmnagement agent that further includes knowledge retrieval and skill management in the reflection waanooon--aannnnn--aeee action agent, following (Wang et al., 2023).
agent
Metrics and Implementation Details. For each game, we report the normalization score rather than the absolute score, i.e., the game score is normalized by the maximum game score. We report the average score of 3 to 20 trials for each game. We also report the average score of 3 human novices. More detailed metrics and LLM hyperparameter configurations can be found in Appendices C-N.
5.2 LLM GAMEPLAY PERFORMANCE
Table 3 shows the gameplay performance of LLMs on Orak with default agentic strategies. Overall, proprictary LLMs outperform open- source LLMs across all games in most cases. Gemini-2.5-pro performs the best on average, ranking first in 5 out of 12 games, with the best average ranking of 3.5. gpt-5/4o forms the second-best group with an average ranking of 3.6. GPT-5 shows superiority in puzzle games, i.e., Baba Is You and 2048, which require strong mathematical and logical reasoning. and spatial understanding abilities. Most small open-source LLMs,, size under 8B, show almost zero score on complex games, i.e., Pokémon-red,, Minecraft, Stardew Valley, StarCraft II, and Slay the Spire, Mid-sized open-source models, Llama3.3-7055 and Qwen2.5-72B, show advances in moderately hard games, such Stardew Valley and Slay the Spire, but their performance remains far below that of proprietary LLMs. See Appendices C-N for more detailed results on each game.
5.3 LLM ARENA
Among 12 games in Orak, Street Fighter III and StarCraft 1l support two-player competitive modes. For Street Fighter III, we conduct pairwise battles among 8 L.LMS with 'zero-shot" agent. Fach pair
7

---

Published as a conference paper at ICLR 2026
Models
Elo
Rank
Medels
Ull
Rank
Lhma-33--1
1358.4
Liana 20
142.9
Quen-2.5-7B
1415,4
Quan-25-77
14452
Miniioon8R
1689.1
1536.8
GPT-le-mini
1453.0
UPI-domiiii
GPTT-l
GPT40
1525.0
of-mini
1602.9
15000
Gemini2.5 2.5-r-
1453.0
Gcnini-2 Sppp
1512.3
Cloude-3,7
1434.3
Clusde-3.7
1561.4
(a) Street Fighter III Arena.
(b) StarCraft II Arena.
Figure 4: Match outcomes and Elo ratings for LLMs in two competitive environments.
Models
Ageut
Action
Adventure
RPG
Simulation
Suntegy
Puzzle
Arg
Strategics
SuperMario
Acedttooney Herbhory
Pokémon Darkeet
Minecroft
Sundew
StorOri22 SlaySpire
BoboleYon
2048
Zaveshot
13.3as
31.2as
4.2s11
47,5am00
0.
Odlena
30/1
0,3ea
1LMA-33
Refication
Planning
300
27.0za+
32,4
5.2
56.3at1
0.0cat
0.0g03
0.0eaa
20.1-.
0.fess
16.7m
91.8ami
54gms
0.gmn
57.0gux
00per
0.0gr
011a0
Zeroshor
29.7asa
29.6.ss
64.2.12
33.3at
93,4
0.0aa
34.5.au
50.0mm
24.7.aa
5.6a1a
GPT-4a
Refleztion
23.3
32.3
20 Auns
65.2am
500
130
50.0pm
Rof-Plan
23.3
91.6eta
5000es
100.0ero
Table 4: Ablation study for agentic modules. 'Ref-Plan' refers to the 'Reflection-Planning agent.
competed in three rounds, and the agent winning 2 out of 3 rounds was declared the winner. To ensure a fair comparison, both agents were assigned the same character, Ken, in all matches. Figure 4(a) shows the relative win rates and Elo ratings. Interestingly, different from the result in Section 5.2, Minitron-8P -BB consistently outperforms all other LLMs and shows the best Elo rating. This may imply that when multiple agents are involved in the environment, adversarial actions can change the game dynamics. For StarCruft II, we conduct pairwise battles among 7 LLMs, with each pair competing in a single round. Both agents were assigned the same race, Protoss, in all matches. As in Figure 4(b), Claude-3.7 3.7oonettt shows the best Elo rating, while GPT-4o and Minitron-88 -B form the second group.
5.4 ABLATION STUDY FOR AGENTIC MODULES
Table 4 shows the ablation results of .lama-332--B and GPT-4o across 4 agent strategies. Interestingly. the impact of adding agentic modules to gameplay performance differs between the two LLMss For GPT-4o, the inclusion of agentic modules consistently improves gameplay performance; 'reflection- planning' agent achieves the best average ranking of 2.2, followed by 'reflection' and 'planning' agents with rankings of 3.1 and 3.3, and 'zero-shot' agent with the lowest ranking of 3.4. However, Llama-3.22 2-BB does not follow this trend. The 'reflection' agent shows the highest average ranking of 5.6, while 'reflection-planiing indicates that, for relatively smaller LLMs like Llama-3.2-3B. adding agentic modules may increase the complexity of the prompt, hindering their decision-making accuracy. These results suggest that the optimal agentic strategy may depend on the inherent capability of the LLM. See Appendices C-N for more detailed ablation analysis for each game.
follows with 6.1, although it adds the planning module. This
5.5 EFFECT OF VISUAL INPUT
Setup. To study the effect of visual input, we divide games into two groups: Group 1 (Table 5) includes games where the provided textual game state can entirely be derived from a visual screenshot. Group 2 (Table 6) comprises games where the textual state contains information beyond what is visible in the current frame (e.g., abstracted inventory lists or off-screen character/item details); for this group, exclude evaluation on Image-only input types for faimess. Minecraft was excluded since the environment does not provide the visionary view of bot that LLM plays.
Result. As shown in Table 5, relying solely on Image-only input led to substantial drop in performance. This was consistently reflected across all models, with average ranks deteriorating significantly compared to Text-only input. In contrast, as shown in Table 5& 6, utilizing Both text and visual inputs produced mixed effects on game scores and model ranks. For instance, in games like Street Fighter III (Group 1), since on-screen visual details are challenging to fully convey textually, adding visual context significantly benefited Claude, increasing its score by 16.6. Conversely, in narrative-heer heavy games such as Ace Attorney (Group 2), the same approach often proved detrimental; GPT-4oos score, for example, dropped by 31.8, and its average rank in that group fell from 2.9
8

---

Published as a conference paper at ICLR 2026
Modzb
lepus
SF3
Seperhlano
Stanlew
Babalayio
2046
Kask
Q-772
Tec:
133
12.4
Ieput
AceAttrrryy
Herboory
Darkest))
Stay5pre
Bock
Vas:
3.3u
25.6wes
15
0/
200
644m
12.
Quea25:7B
Borh
Tox
17.
10.0ace
2.5
32
2/8
2.
82.7.sx
31,8
0.
0.8g0
9.5
9.1
0we225-778
Bei
383
7..7
369
0.7,m
34
Q-3en25-3328 5-3288
Buth
Test
68.0gm0
15:4uz
17.0gss
35.0u
27.8
91.8,3
89.9.5
0.
0.8g0
0.8
107
414..
C271-40
Imuge
Ustt
46.3
27
41
210
62
3D
GPT-4
40.6-29
41.7xx
50.0
236.
G.
Imag
183
28.5
20
58
74
Geztini 2.
Bork
Text
567.14
5g
67.8
04.9
K333re
KM.3re
91.2g1
M57
108.
108.
51
76 26
21
522
Bott
20
48.9-a
68/
4.0
Clande
Tex
71.3
S1.
62.0ua
686ux
722
63.0
89.9
500
500
150
4771s
Claxle3.77
Taage
33.3
33
20 Dace
8duas
8.0
Bod.
333
22.6aa
49.8ur
20 Dace
6.7as
6.2
Table 6: Modality comparison (Group 2).
Table 5: Modality comparison (Group 1).
Models
Finetune
Intra-Game
OOD-Game
Non-Game
SF3
DarkestD
StaCralt2
SlaySpire Baballs You
SuperMario
2048
Math500
WebShop-E WebShop-H H
0,0eaa
0.0sss
0.0ass
0.0zwo
20.0:a3
18.7
0.1
24.1
2.7.xs
L.ama-3.2-1B
42.0
93.42z
0.0ass
8.0
20.0.a3
26.7am2
2.8
25.6
0,5gm
0.0ga0
Llama-3.2-3B
12.0
87.2
0.0
0.0
20.0-a3
31.
0.1
29.6
0.0
0.0eaa
40.0
92.0aaz
0.0
10.7p1a
20.0
34.4
3,1
36.1
8.4
126::1
Table 7: Generalization performance of LLMs fine-tuned on expert gameplay trajectories from Orak.
(Text-only) to 5.0. This highlights that the impact of combining modalities varies considerably, with some scenarios showing improved ranks or scores while others demonstrated a decline.
5.6 EFFECT OF FINE-TUNING
Setup. To study the effect of fine-tuning, we consider three types of generalization: Intra-game, OOD-game, and Non-game. Intra-game generalization evaluates whether an LLM can adapt to unseen scenarios within the same game, e.g., new stages or characters. OOD-game -g-ae generalization evaluates whether fine-tun tine-tuning a specific set of games enables the model to act better on unseen games. Non-game game generalization evaluates whether our gameplay fine-tuning set helps the model to perform better on non-game tasks like math or web navigation. For Intra-game and Non-game generalizations, we fine-tune L.MMs on all 12 games, while for OOD-game generalization, we split 12 games in 10 training games and 2 test games, i.c., Super Mario and 2048. Additionally, to study the effect of data quality and scaling, we perform fine-tuning on high-score (original), low-score (samples with the lowest game score), and their mixed datasets (doubling the data volume). See Appendix for detailed fine-tuning configurations and unseen scenario details in intra-game generalization.
Intra-game Generalization. As shown in Table 7, fine-tuned Llama-3.2- 1BBBB show better gener- alization to unseen scenarios than the pretrained ones; in 3 out of 5 games, both fine-tuned models outperform their pretrained counterparts, The performance gain largely comes from the model learn- ing to generate valid actions more reliably after fine-tuning, especially in environments where the pretrained model frequently failed to act meaningfully. However, this approach appears insufficient to significantly enhance the spatial reasoning abilities of smaller models. This is evident in Baba Is You, where fine-tuned models struggle to construct the winning condition required to score above 20.0. Also, since StarCraft II requires complex strategic reasoning to defeat enemies, the strategic patterns learned from the fine-tuning set were not sufficient for the model to win on new maps.
00D-game Generalization. Similarly, despite being trained exclusively on trajectories from different games, fine-tuned models perform significantly better on OOD games like Super Mario and 2048. This suggests that fine-tuning on trajectories shaped by reflection and planning enables models to learn transferable greision-mmkknne routines, as the underlying capabilities required for these behaviors are shared across games. Specifically, the improvement in 2048 likely benefits from its structural similarity to Baba Is You, a training game that also uses 2D grid layout and discrete move actions (up, down, left, right). More fine-tuning analysis with varying OOD gaps in Appendix Q.
Non-game Generalization. More interestingly. LLMs fine-tuned on our gameplay trajectories sometimes show improved performance in non-game tasks like Math500 and WebShop (Yao et al., 2022). In Table 7, LLaMA-3.2-3B consistently improves on both Math500 and Webshop. In particular, on WebShop-E (easy) and -H (hard), it shows a dramatic gain from 0.0% to 8.4% or 12.6%. indicating our fine-tuning set is more effective for agentic tasks that require decision-making abilities similar

---

Published as a conference paper at ICLR 2026
Daora
SF3
DarkestD
SturCraf2
SlaySpire
BabalsYon
Model
Ponsed
Hard (Original)
Real-sime
Paused
Easy
Real-time
line wninge
12.021100
87.2=95
0,0000
00000
20,0000
high (1oo)
40.027.1
92.02002
0.0000
10.7&:12
20.02:00
gpt-4-mini (19.6 sec)
73 75.0+000
0.000
100,0:00
0.00000
(110K)
18.0::64
40.0000
0D000
20.0+00
(27.2 see)
100.0:20
00+00
100.0:00.0
100.0:00
mixed (220K)
18.0=148
40.0=00
00000
10.7+11
20.0+00
gemini- 2-por (99.2,00)
100.0190
(a) Effect fine- munittunng data quality and scale.
(b) Real-time game performance on StarCraft
Table 8: Effect of (a) fine-tuning data quality/scale and (b) latency-aware real-time evaluation.
to gameplay. In contrast, the smaller model, Llama-3.2-10 shows no observable generalization to non-game tasks, which suggests that the extent of generalization may depend on model capacity.
Effeet of Data Quality and Scaling, Table 8(a) shows the effect of data quality and scaling in fine-tuning generalization. Overall, high-score dataset alone the most effective, while the low-score dataset contributes little or even degrades performance in certain cases (e.g., Darkest Dungeon). Simply scaling up the dataset by mixing high- and low-score samples does not yield clear improvement, indicating that data quality, rather than sheer quantity, plays more critical role achieving better generalization.
5.7 LATENCY-AWARD EVALUATION IN REAL- TIME GAMEPLAY
Among the 12 games in Orak, games, including Street Fighter III, Super Mario, and Starcraft II, require pausing for evaluation. Here, we conduct a latency- evaluation on StarCraft II by enabling real-time gameplay. As shown in Table 8(b), when switching from the paused setting to the real-time mode, all models failed (0 score) under the hard level (the original setup), indicating that liming-sensitive
games are strongly affected by inference latency. When the difficulty level was reduced to easy, all models achieved perfect scores under the paused setting, but only GPT-4o maintained perfect performance under real-time conditions. The average response latencies per step with the reflection-planning agent were 19.6 for GPT-4o-mini, 27.2 for GPT-4o, and 99.2 for Gemini-2.5-pre These results suggest that GPT-4o provides the best trade-off between latency and gameplay accuracy among the three evaluated models.
DISCUSSION
Conclusion. this paper, we introduce Orak, benchmark designed train and evaluate LLM agents across diverse video games. Orak enables comprehensive assessments of LLM capabilities required play most game genres. Through a plug-an plug-and-pla and-play interface powered by MCP, it allows consistent evaluation of rapidly evolving LLMSs over various agentic modules. In addition, we release a fine-tuning dataset game interaction trajectories of top-performing LLMs, which can effectively transform pre-trained LLMs into gaming agents. With the comprehensive game set and user-friendly interface, Orak sets a new foundation for game-based LLM evaluation, driving progress towards versatile and high-performin gaming agents.
Limitations. We note that Orak provides environments favorable for LLM reasoning and decision- making by pre-processing game states into structured text where information unnecessary to gameplay is hidden. This may offer insights for providers seeking to deploy resource-efficient LLMs games. We leave a complementary direction, providing the full in in-game states with rich uncurated texts to LLMs, for future work. Further discussions, including cost and licensing, provided in Appendix U.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 0rXiv:2303.0877 2023.
Jaewoo Ahn, Junseo Kim, Heeseung Yun. Jachyeon Son, Dongmin Park, Jaewoong Cho, and Gunhee Kim. FlashAdventure: A benchmark for GUI agents solving full story ares in diverse
10

---

Published as a conference paper at ICLR 2026
adventure games. In Christos Christocduulpooulss, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 23354 23384, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-8917668 76-332-6. doi: 10. (0.18833//1/2000 12233/1/1225.aa// e20225.nnnllmmain. 1192. URL :ttps: //aclantho ogy- o org/ 2.25. emn m-nam i
92/.
Anthropic. Claude 3.7 sonnet: Our most capable model yet. wtwps//claddee---soooee
2025. Accessed: 2025-09-08.
Baekalfen. Pyboy: Game boy emulator written python. httpps:///t wig.h..h........ wtt.cccc//////.e BaanmBaakalll fen/
PyBoy, Accessed: 2025-09-23.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Sam Barlow. Her Story. https::/www.herstoo./
com, 2015. Accessed: 2025-09-22.
Blizzard Entertainment. StarCraft II. https://starcraatttt com, 2010. Accessed: 2025-09-22.
Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3-4):324-345, 1952. doi: 10.2307/2334029.
Bug Kioocht. BaseMod.
com/ sharedfiles/
1 2018a Accessed: 2025-09-22.
2018a.
Bug Kioocht. Mod the Spire. https://tteamcoom//
wwt.coo/////a//oe htt.s/haareeeess//////e 2018b. Accessed: 2025-09-22.
BurnySc2. Starcraft II Bot API Client Library for Python 3.
https://github.com
BurnySc2/python-sch 2017. Accessed: 2025-09-21.
Capcom. Street Fighter III: 3rd Strike. pttpp::/strretfiine
candomm.com/we wiki/
StrrettFFighte_nis :_dd:_Ir__:rrrreeeeee
1997. Accessed: 2025-09-22.
Capcom. Phoenix Wright: Ace Attomney
Phoenix_Wright: Aaaptt::AAA_AA_onee__e 2001. Accessed: 2025-09-22.
Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. Fireact: Toward language agent fine-tuning. arXiv preprint arXiv:23100055915 2023.
Kevin Chen, Marco Koltun, and Philipp Krähenbühl. Reinforcement learning for long-horizon interactive 1lm agents. arXiv preprint 2502 2025.
Brody Huval, Alcksei Petrenko. Jackson Hamburger, Vladlen
Zchui Chen, Kuikun Liu, Qiuchen Wang, Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, and Feng Zhao. Agent-flan: Designing data and methods of effective agent tuning for large language models. arXiv preprint 24x. 2024.
Christian Kauten. Super Mario Bros for OpenAl Gym. https: http//gttttttttttt teritthb.coom com Kaamm//attennna ja/ cern-super-aee penoer-maroobbee 2018. Accessed: 2025-09-22.
Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang. Zhijie Yan, Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919 2023.
Gabriele Cirulli. 2048. https://play204: co/, 2014. Accessed: 2025-09-22.
ConcernedApe. Stardew Valley. https:ttp//// /ttp///wwww//m www. www.stardwwaalll letddeewwwwllleeeee...le. ley. net, 2016. Accessed: 2025- 09-22.
BepInEx Contributors. Bepinex: Unity xna game patcher and plugin framework. https:// gi tit... wtt.c/omm/eeeee wwcm//BpTnnx//epppieeee Rnnnxxx, 2025a. Accessed: 2025-09-21.
11

---

Published as a conference paper at ICLR 2026
PrismarineJS
contributors. Prismarine/S/mineetaayee Create Minecraft bots with powerful, stable, and high-level JavaScript API. https://githin. www//oo//////////oo Pri smar neJs meeeminneeeeeeeeee flayer, 2013. Accessed: 2025-09-22.
Unity Doorstop Contributors, Unity doorstop: A tool execute managed .net assemblies inside unity. https: //gtt///g/hhb////// com/ /en//Niigg NeighTools httro/ooo/o//ittttorsssoeeoeeeee UnityDoorstop. 2025b. Accessed: 2025-09-21.
Authony Costarelli, Mat Allen, Roman Hauksson, Grace Sodunke, Suhas Hariharan, Carlson Cheng, Wenjie Li, Joshua Clymer, and Arjun Yadav. Gamebench: Evaluating strategic reasoning abilities 2024.
of Ilm agents. arXiv preprint arXiv:2406.0660
Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao, Ziqiao Meng. Guangyan Zhang, Qichao Wang, Yiwen Guo, and Irwin King. Recent advances in speech language models: A survey. arXiv preprini arXiv:2410.0375/ 2024.
DIAMBRA. DIAMBRA: Reinforcement Learning Platform for Competitive Video Games. https: //www. , di ambra a 2025. Accessed: 2025-09-22.
Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang. Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. Advances in Neural Information Processing Systems, 35: 18343-18362, 2022.
Peiyuan Feng, Yichen He, Guanhua Huang, Yuan Lin, Hanchong Zhang, Yuchen Zhang, and Hang Li. Agile: novel reinforcement learning framework of Ilm agents. arXiv preprint vrXiv:2405.1777 2024.
Xidong Feng, Yicheng Luo, Ziyan Wang. Hongrui Tang, Mengyue Yang, Kun Shao, David Mguni, Yali Du, and Jun Wang. Chessgpt: Bridging policy learning and language modeling. Advances in Neural Information Processing Systems, 36:7216-7262, 2023.
R. A. Fisher. Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh, 1925. 1st ed.
Forgotten Arbiter. Communication Mod. https://gi wwi.ghttuub.... / ttt.bb.ccm//orrgttteetbtttteeerr
Communicationoen 2019. Accessed: 2025-09-22.
Game Freak. Pokémon Red Version. https: /pok//oooooo...moo fandom.com/w httm//wwk//ooo/3
4.9mon_Red_and L willueeeeerionsseeee 1996. Accessed: 2025-09-22.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama herd of models. arXiv preprint arXiv:2407.21788 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint 1.XXi:25511.194:: 2025.
Danijar Hafner. Benchmarking the spectrum agent capabilities. arXiv preprint 170.022009.06780, 2021.
Matthew Hausknecht, Prithviraj Ammanabrolu, Marc-Alexandre Côté, and Xingdi Yuan. Interac- tive fiction games: colossal adventure. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 7903-7910, 2020.
Hempuli. Baba is you. https:t///// httmpheeppll..o/ www.baba/ 2019. Accessed: 2025-09-22.
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. Measuring coding challenge competence with apps. arXiv preprint 100.009.0
09938, 2021.
Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. Model context protocol (mep): Landscape, security threats, and future research directions. arXiv preprint arXiv:2503..3227, 2025.
Lanxiang Hu, Qiyu Li, Anze Xie, Nan Jiang, lon Stoica, Haojian Jin, and Hao Zhang. Gamearena: Evaluating Im reasoning through live computer games. arXiv preprint 1rXiv:2412.06394 2024a.
12

---

Published as a conference paper at ICLR 2026
Lanxiang Hu, Mingjia Huo, Yuxuan Zhang, Haoyang Yu, Eric P Xing, lon Stoica, Tajana Rosing, Haojian Jin, and Hao Zhang. Imgame-bentt bench: How good are llms at playing games? arXiv preprint 1rXii:2505.15146, 2025.
Sihao Hu, Tiansheng Huang, Fatih Ilhan, Selim Tekin, Gaowen Liu, Ramana Kompella, and Ling Liu. A survey on large language model- based game agents. arXiv preprint arXiv:2404.02033 2024b.
Sihao Hu, Tiansheng Huang, and Ling Liu. Pokéllmon: A human-parity agent for pokémon battles with large language models. arXiv preprint arXiv:2402.01118, 2024c.
Jen-tse Huang, Eric John Li, Man Ho Lam, Tian Liang, Wenxuan Wang, Youliang Yuan, Wenxiang Jiao, Xing Wang, Zhaopeng Tu, and Michael R Lyu. How far evaluating Ilms' gaming ability in multi-agent environments. arXiv preprint arXiv:2403. 1807, 2024.
the decision-making of Ilms?
Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International conference on machine learning, pp. 9118-9147. PMLR, 2022a.
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05600, 2022b.
Christian Kauten. Super Mario Bros for OpenAl Gym. https: //gg/ggthhhaa com/ wwwe/co//////eeeeeeeee Kautenjal ja/ gym-super-mariom
2018. Accessed: 2025-09-21.
Kgleken. Darkestdungeenbot. 2023. Accessed: 2025-09-21.
com http://wwww..oooeeeeeeeeeeeeeeee
Rahima Khanam and Muhammad Hussain. Yolov 11: An overview of the key architectural enhance- ments. arXiv preprint arXiv:2410. 17725, 2024.
Raph Koster. Theory fun for game design. O'Reilly Media, Inc.", 2013.
Heinrich Küttler, Nantas Nardelli, Alexander Miller, Roberta Raileanu, Marco Selvatici, Edward Grefenstette, and Tim Rocktäschel. The nethack learning environment. Advances in Neural Information Processing Systems, 33:7671-7684, 2020.
Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jic Wang, Xiuyi Chen, et al. From system to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.14419, 2025.
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating 1lms agents. arXiv preprint arXiv:2308.000 036888 2023.
Weiyu Ma, Qirui Mi, Yongcheng Zeng, Xue Yan, Runji Lin, Yuqiao Wu, Jun Wang, and Haifeng Zhang- Large language models play starcraft ii: Benchmarks and chain of summarization approach. Advances in Neural information Processing Systems, 17:133386-133412 2024.
MegaCrit. Slay the Spire. https: httpps:/www..eggattt http://wwwwggaart.tcc/ com, 2017. Accessed: 2025-09-22.
Mojang Studios. Minecraft. https: http:///wwwwwwnnn.atttctmm net, 2011. Accessed: 2025-09-22.
Peter Mühlhacher, Nikos I Bosse, and Lawrence Phillips. Towards a realistic long-term benchmark for open-web research agents. arXiv preprint arXiv:2409.14913, 2024.
NVIDIA.
Introducing nvidia ace for games spark life into virtual characters with generative ai.
hetps://www. nvidia. cwm/co///eeeoeeeo sen-us/ge wtt.g/ffoo///oo//ooooeooooc neww// wwwrlll-aee-----gggg--eeeeerreie
anterratvvvvaiinpss/..eeoa 2025. Accessed: 2025-09-22.
Davide Paglieri, Bartlomiej Cupiat, Samuel Coward, Ulyana Piterbarg, Maciej Wolezyk, Akbir Khan, Eduardo Pignatelli, Lukasz Kuciński, Lerrel Pinto, Rob Fergus, et al. Balrog: Benchmarking agentic Ilm and vlm reasoning on games. arXiv preprint 2xXiv:2411.13544 2024.
13

---

Published as a conference paper at ICLR 2026
Yichen Pan, Dehan Kong, Sida Zhou, Cheng Cui, Yifei Leng, Bing Jiang. Hangyu Liu, Yanyi Shang, Shuyan Zhou, Tongshuang Wu, et al. Webcanvas: Benchmarking web agents in online environments. arXiv preprint arXiv:2406. 12373, 2024.
Andreas Pardeike. Harmony: A library for patching, replacing and decorating .net and mono methods during runtime. https: ht///gtthuu wow http/aaadeiie /aaeeeaaroo....aa 2025, Accessed: 2025-09-21.
Joon Sung Park, Joseph O' Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pp. 1 1-22, 2023.
Pathoschild. SMAPI Staardew Modding API. https: http:///mmap.... https::/smapi.io// io/, 2025. Accessed: 2025-09- 22.
Archiki Prasad, Alexander Koller, Mareike Hartmann, Peter Clark, Ashish Sabharwal, Mohit Bansal, and Tushar Khot. Adapt: As-needed decomposition and planning with language models. arXiv preprint arXiv:2311.05772 2023.
Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov, Agent q: Advanced reasoning and learning for autonomous ai agents. arXiv preprint arXiv:2408.07190 07999, 2024.
Siyuan Qi, Shuo Chen, Yexin Li, Xiangyu Kong, Junqi Wang, Bangcheng Yang, Pring Wong, Yifan Zhong, Xiaoyuan Zhang, Zhaowei Zhang, al. Civrealm: A learning and reasoning odyssey in civilization for decision-makiin agents. arXiv preprint arXiv:2401. 10568, 2024.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D) Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:57728-53741, 2023.
Red Hook Studios. Darkest Dungeon. http::://wwwwwaaat httpss//ww..arrkettddngee com, 2016. Accessed:
2025-09-22.
Robojumper. Darkest dungeon save editor. https://githui wtt www.:ooooooooooooooooooooo jumper/ Darkesstunggonsavedddtcc 2023. Accessed: 2025-09-21.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634-8652. 2023.
Sharath Turuvekere Sreenivas, Saurav Muralidharan, Raviraj Joshi, Marcin Chochowski, Ameya Sunil Mahahaleshwarkar. Gerald Shen, Jiaqi Zeng, Zijia Chen, Yoshi Suhara, Shizhe Diao, et al. Llm pruning and distillation in practice: The minitron approach. arXiv preprint arXiv:2408. 1796, 2024.
Weihao Tan, Wentao Zhang, Xinrun Xu, Haochong Xia, Ziluo Ding, Boyu Li, Bohan Zhou, Junpeng Yue, Jiechuan Jiang, Yewen Li, al. Cradle: Empowering foundation agents towards general computer control. arXiv preprint 24Xiv:2403.03186 2024.
Wenjie Tang, Yuan Zhou, Erqiang Xu, Keyan Cheng. Minne Li, and Liquan Xiao. Dsgbench: A diverse strategic game benchmark for evaluating 11m- based agents in complex decision-making environments. arXiv preprini arXiv:2503. 06047, 2025.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiss Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew Dai, Anja Hauth, Katic Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.1 1805, 2023.
14

---

Published as a conference paper at ICLR 2026
Chen Feng Tsai, Xiaochen Zhou, Sierra S Liu, Jing Li, Mo Yu, and Hongyuan Mei. Can large language models play text games well? current state-of-th ffte-of-the-ane art and open questions. arXiv preprint arXiv:2304.02886 2023.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.1629), 2023.
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024a.
Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better Ilm agents. In Forty-first International Conference on Machine Learning, 2024b.
Xinyu Wang, Bohan Zhuang, and Qi Wu, Are large vision language models good game players? arXiv preprint arXiv:2503.02358 2025a.
Zihan Wang. Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Monica Lam, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, and Manling Li. Ragen: Understanding self-evolution in IIm agents via multi-turn reinforcement learning. arXiv preprint arXiv:2504.20073. 2025b. URL https: aawiv org /250 20073.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 55:24824-24877, 2022.
B. Welch. The generalization of "student's" problem when several different population variances are involved. Biometrika, 34(1-2):28 2):28335, 1947. doi: 10. 10.1093/biomee/34112
Wikipedia contributors. List of video game genres. https: //en wi wwikipedia..rg/viiie Lists_o__vieoe_aaaee geneee, 2025. Accessed: 2025-09-22.
Yue Wu, Xuan Tang, Tom M Mitchell, and Yuanzhi Li. Smartplay: benchmark for llms as intelligent agents, arXiv preprint arXiv:2310.01557 2023.
An Yang, Baosong Yang, Beichen Zhang. Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. 2024.
Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35:20744-20757, 2022.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang. Xiao Liu, Yuxiao Dong, and Jie Tang. Agenttuning: Enabling generalized agent abilities for lims. arXiv preprint arXiv:2310. 12823, 2023.
Siliang Zeng, Quan Wei, William Brown, Oana Frunza, Yuriy Nevmyvaka, and Mingyi Hong. Reinforcing multi-turn reasoning in Ilm agents via turn-level reward design. arXiv preprint arXiv:2505. 2025. URL https:/ http///axivvv//// ort. http:/aab/255.11115
Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, and Yisong Yue. Datascibench: An I1m agent benchmark for data science. arXiv preprint arXiv:2502.1389) 2025.
Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei. Lim as a mastermind: A survey of strategic reasoning with large language models. arXiv preprint 244 2024.
15

---

Published as a conference paper at ICLR 2026
Xiangxi Zheng, Linjie Li, Zhengyuan Yang. Ping Yu, Alex Jinpeng Wang, Rui Yan, Yuan Yao, and Lijuan Wang. V-mage: game evaluation framework for assessing visual-centric capabilities in multimodal large language models. arXiv preprint arXtv:2504.06148, 2025.
Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al. Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions. arXiv preprint arXiv:2406.15877 2024.
16

---

Published as a conference paper at ICLR 2026
Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Games
(Supplementary Material)
A GAME GENRE CATEGORIZATION
Video games are generally categorized into six widely recognized genres, i.e., Action, Adventure, Role-Plan Dole-Playing, Simulation, Strategy, and Puzzle, each characterized by distinct gameplay struc- tures (Wikipedia contributors, 2025). Action games emphasize responsiveness and precise physical control. Adventure games focus on narrative exploration, interactive dialogues, and clue-based progression grounded in logical inference. Role-Playing games focus on character progression, stat-driven combat mechanics, and quest-based narrative development. Simulation games present restem-driven environments in which players manage complex and interdependent variables such as time, resources, and procedural systems. Strategy games are a genre that emphasizes planning, resource management, decision-making, and tactical execution. Puzzle games revolve around rule- based problem solving, pattern recognition, and logical or spatial reasoning, typically within clearly defined system of constraints.
In Table 9, we categorize each of the 12 video games in Orak into one of the six major genres based on the primary gameplay characteristics and the genre information provided by the respective game publishers, Some games belong to one or more genres due to hybrid gameplay mechanics.
Game
Action
Adventure
Role-Playing
Simulation
Strategy
Puzzle
Street Fighter III
Super Mario
O
Ace Attomne
O OOD
Her Story
Pokémon Red
COO
Darkest Dungeon
A
A
Minecraft
8
Stardew Valley
A
A
StarCraft II Slay the Spire
A
8
Baba Is You 2048
8
Table 9: Genre categorization of the 12 games in Orak. O denotes the main genre and A indicates the secondary genre.
(a) Street Fighter III (Capcom, 1997) is classified as an Action game, as it primarily relies on responsiveness, frame-precise input, and physical dexterity. Its gameplay requires fast reflexes and mastery of complex input sequences.
(b) Super Mario (Christian Kauten, 2018) categorized as Action game. The core gameplay emphasizes precise timing in jumping and movement, demanding moment-to-moment control in response to environmental hazards and enemy placements.
(c) Ace Attorney (Capcom, 2001) is labeled as an Adventure game due to its narrative-driven structure, reliance on clue collection, and logical deduction. Players progress by interacting with characters and uncovering story elements through investigative mechanics, with minimal emphasis reflex-based input.
(d) Her Story (Barlow, 2015) similarly fits within the Adventure category. Though more experimental in form, it shares a strong focus on narrative discovery through a search-based interface, requiring players to piece together fragmented story using non-linear exploration and deductive reasoning.
(e) Pokémon Red (Game Freak, 1996) is classified as a Role-Playing game, as its primary mechanics involve turn-based combat, character progression, stat management, and quest-driven exploration.
17

---

Published as a conference paper at ICLR 2026
Additionally, Adventure assigned as secondary genre reflect the game's emphasis world exploration, interaction with non-player characters, and sequential progression through narrative landmarks.
(1) Darkest Dungeon (Red Hook Studios, 2016) is similarly assigned Role-Playing as the main genre, supported by stat-driven character development and progression systems. However, it also includes significant Strategy elements, as players must carefully manage resources, form party compositions, and make tactical decisions in turn-based combat. Adventure was further added a secondary genre due to its dungeon-crawling structure and emphasis on risk-driven exploration.
Simulation game due its open-ended, system -ystem-driven mechanics, including resource gathering, crafting, and environmental manipulation. also exhibits Role-Playing traits through its progression systems and player-driven narrative development, warranting secondary classification.
(g) Minecraft (Mojang Studios, 2011) 1) is categorized
(h) Stardew Valley (ConcernedApe, 2016) is also assigned to the Simulation genre based on its emphasis on time management, farming systems, and interrelated mechanics spanning multiple in- game variables. It incorporates Role-Plos Playing through relationship-buildin and character progression, and Adventure through dungeon exploration, seasonal events, and quest-based interactions.
(i) StarCraft (Blizzard Entertainment, 2010) is classified as a Strategy game, consistent with its real-time strategic planning, resource allocation, and micromanagement mechanics. Given the importance of unit-level control, Action was added as a secondary genre to reflect the real-time, reflex-driven demands during gameplay.
(i) Slay the Spire (MegaCrit, 2017) categorized solely as a Strategy game. The gameplay centers around deck- route optimization, and turn-based combat, requiring players to plan several moves.
(k) Baba Is You (Hempuli, 2019) is clearly identified as a Puzzle game, as its mechanics are centered on solving logic-based problems through the manipulation of in-game rules represented by words. The core loop involves constrained, rule-based problem solving and spatial reasoning.
(I) 2048 (Cirulli, 2014) is also classified as a Puzzle game, characterized by deterministic mechanics, arithmetic pattern recognition, and constraint-based spatial logic within fixed grid.
This genre classification provides a structured foundation for analyzing agent cognition and gameplay dynamics across diverse games.
B REQUIRED LLM CAPABILITIES FOR GAMEPLAY
Looking at the Figure 3, most games demand advanced LLM capabilities across multiple dimensions, as they are designed to challenge a wide range of human cognitive skills. (1) Action games, Street Fighter III and Super Mario in red, require spatial reasoning and rule following, more than long-context understanding and planning. (2) Adventure games, Ace Attorney and Her Story in yellow, emphasize long-tex understanding and logical reasoning due the need comprehend long storylines. (3) Role-playing games, Pokémon Red and Darkest Dungeon in brown, require strong long-term planning, logical reasoning, and rule-following abilities to understand game-specific rules and complete milestones of game tasks. (4) Simulation games, Minecraft and Stardew Valley in green, also require high levels of long-term planning and rule-followig abilities. While Minecraft requires strong spatial reasoning and error handling, which are generally essential for simulation games, Stardew Valley gets lower scores for them because these abilities are not critical for its evaluation task; carning money by harvesting crops. (5) Strategic games, StarCraft Il and Slay the Spire in blue, require various LLM abilities for gameplay. Notably, these two games are the only ones in Orak that require 5 different L.LM capabilities rated at level 3, highlighting that recent strategic video games increasingly demand a wide range of cognitive skills for effective gameplay. (6) Puzzle games, Baba Is You and 2048 in purple, require high levels of spatial reasoning, logical reasoning, and long-term planning because puzzle games are typically designed to require complex problem-solving through multiple reasoning hops and spatial understanding.
18

---

Published as a conference paper at ICLR 2026
C STREET FIGHTER III
C.1 GAME DESCRIPTION FOR STREET FIGHTER III
Street Fighter III (Capcom, 1997) is a 2D compet- itive fighting game, known for precise controls, deep mechanics, and a diverse roster of characters. Each character features unique moves, combos, and super arts, requiring precise timing and strategic decision-makinn Players aim to defeat their opponent through a mix of normal attacks, special moves, and advanced mechanics like parries and cancels. For implementation, we use Diambra Arena environment (DIAMBRA, 2025), a Docker-based platform designed for RL research. Street Fighter III is one of the environments sup- ported by Diambra, which not only enables seamless extraction of the game state-such forward interface for sending controller inputs. In this setting, the agent operates in a discrete action space that directly corresponds to various controller actions, including directional movement, attack buttons, and their combinations, enabling intuitive and fine-grained control of the in-game character.
Environment.
(a) Ken.
(b) Chun-Li.
Figure Two of playable characters in Street Fighter III.
screenshots, health, timer, and super bar values also provides a straight-
The game supports both Magle-player and multi-player modes. In single-player mode, the player progresses through ten increasingly difficult stages, facing stronger opponents at each level. Each stage follows a best-of-three format, where the player must win two out of three matches to advance to the next stage. The game ends upon either completion of the final stage or defeat. In multi-player mode, two players compete in best-of-three match, and the game is over once winner is determined. For our default evaluation setting, agents play Ken in both modes (see Figure 5(a)). However, since the environment supports variety of characters, we also conduct evaluations using Chun-Li to demonstrate intra-game generalization capabilities (see Figure 5(b)).
Observation-to-Text Conversion. The Diambra environment offers a convenient interface for ex- tracting the game state from Street Fighter III. Through this interface, we obtain the latest game frame at a resolution of 224x384, along with key state information such remaining time, player and opponent health, super bar gauge, super count, stun bar gauge, and stun status. However, criti- cal aspect in fighting games is understanding the relative positions of the characters, which is not directly provided by the Diambra environment. To address this limitation, we employ lightweight YOLOv1I object detection model (Khanam & Hus- sain, 2024) to extract the relative positions of the two characters from the game frame. Based on the computed distance, we classify the spatial rela- tionship into three discrete categories-vers close, close, and far-and incorporate this information into the agent's user prompt.
00
95
Toiless
NEWY
teddy bear 0.59 person 0.55
Figure 6: Character detection using YOLOv11 model (Khanam & Hussain, 2024) in Street Fighter III.
Action Space. the Diambra environment, the native action space is defined on a per-frame basis and consists of 18 discrete actions. These are composed of:
Idle action (1 total): Idle (No action)
Movement actions (8 total): Left, Upt Up. UigRight, Right, Right+ Down,
LLff+
Attack actions (9 total): Low Punch, Medium Punch, High Punch, Low Kick, Medium Kick, High Kick, Low Low Kick, Medium - Kick, High Punch High Kick
To enable more strategic and temporally consistent behavior, we use a liveer-level action space that abstracts these frame-level controls into semantically meaningful commands. Each high-level action is mapped to a predefined sequence of low-level controller inputs, often spanning multiple frames. The high-level action space is divided into two categories:
19

---

Published as a conference paper at ICLR 2026
System prompt (Planning)
User prompt (Planning)
helptul assistant intograted "Street Fighter IIII 3rd Strike PC, aesisting future decisioneaaking Your goal is to aseist long-term strategy planiing defeat oppenent. Rased the turget task and the playa's current propose
tNA Target task Defeat the opponent
situtioon. responses must be procise, concrate, and highly relevent the players objectives.
pengress, your
suituble subtesk for
eNA Previons subnsak the task
The current subtask astempt another throw to break the and reduce their health.
Subtaok ressoning: Decide whether previous subtask finished and whether is necensary to prupose a new subuskk The subuask should be straightferward, commibtte the target task and be most suiable for the current situation, which should he completed within fow actions You should respond You
thit Last execuned action
Medium Punch-Medium Kick
Punch Hiigh Kick
with:
finich target 2. What the current progress of target accending to anilysis step 1? Plaase do not make any sssumprion they are not menriond in the ahove information. You should aesume that
mnalyze step by step.
che Self reflection of the last executid action
The attenspted throw break the opponent's i guard and roduce their health unsurcessfull The opponent's health remained unchanged, and the distanee between the players incressed, indicating that the opponen: likely evaded or counteced the throw.
doing the task from scratch.
provious subtask provided) What the provious subeask?
previous subtask finish? Or it impouper for the current situation? Then select new une, ocherwise you should reuse the last subusk.
esa Curcent
playing Ken Street Fighter Your opponent is Yaeg.
You are facing l:ft
Game screenshot
Distance from Opporant: close Remsining:
1900
95
PS PD START
Health:
Your Health 161
YANG
Opponent's Health: 134
Bar
Your Super Bar Gunge: 32
Opponent's Super Bar Guage: 9
Super Count (Count of activated movej:
Super
Opponent's Super Count:
Sunn Bar Giage:
Your Bar Guage:
Opponent's
Guages
IsStanned (Ot not stumned, I: sturnedj:
Your Starus: 0
Oppunent's Special Status:
You MUST respand in the sormat described below, and you should not cutpui comments other information. esa Subaask reasoning
2....
tith Subizsk
The current subtask
Figure 7: Planning prompt for 'reflcttoon-planing agent playing Street Fighter III.
Charsicer-agnostic actions (14 total): Move Closer, Move Away Jump Closer, Jump Away Super Attack, Low Punch, Medium Punch, High Punch, Low Kick, Medium Kick, High Kick, Low Punch+Low Kick, Medium
Kick, High Punch+ Kick
Character-speeffic actions:
- Ken: Fireball (Hadouken), Hurricane Kick, etc.
Chun-Li: Kikkoken, Hyakuretsukyaku, etc.
For example, if the character is positioned the left side of the screen and the high-level action is 'Move Closer', the system issues 'Right' movement commands over four frames. If the action is Tire- ball', the corresponding low-level sequence would be 'Down' "Down Right'+ Punch'. Since the number of character-specinf actions varies, the total size of the high-level action space differs depending character, typically ranging around 20 actions.
'Medium
C.2 GAMEPLAY PROMPT FOR STREET FIGHTER III
Our implementation of Street Fighter supports four types of agents: reflection, planning, reflection- planning, and zero-shot. Among these, introduce prompts for the ceffection-planning agent in this subsection. Figures 7-9 present the prompts used by the reflection-planiiin action inference, and reflection, respectively. At each step, the agent plans to determine the subtask, using the planning module. Based on this subtask, the action inference module infers the optimal action to execute. Finally, the reflection module evaluates whether the executed action was successful.
agent for planning,
Planning prompt. As shown in Figure 7, the system prompt provides detailed instructions for an agent support strategic planning. It defines the assistant's role in proposing suitable subtasks based
20

---

Published as a conference paper at ICLR 2026
System prompt (Action Inference)
User prompt (Action Inference)
You world goal
agaresiive Street Fighter ill: 3ed Strice
suptask
opponent quickly efficiently possible
distance by using forwand dash or advuncing special move, then apply presuue with mix of medium and heavy utteees to reduce the oppozents health.
The careet subtask
mong optimal statepes
Analyse curren game stuee distanee between your chamsoer's the upponeet's clumetes and semsining hoilh, Than, determine nost actions to execune, ensering you maintain offensive pressure. she Stratayy Gaidelines Ehe
saV Last esecuted action:
Madium Purich Modium Kick
1. Yos can only cutgut moat TWW actions in the cutput
High Punch-Eligh Kick
2. Choose the appeoprizte move based on the distance from the opponent and the cumentt situsion.
lest execated thes health unsucceedfil
Salf
5. Unitze defencive evesive techniques to minimizo incoming damage
The unempped tlow
the oppoaeet's graed reduce uppanenn's health remainmal
Combine sttacks special
space and apply pressure
opponent.
unchanged. and the distance berween the players increased. indicating that the oppusent licely ovaded counsered ed the Bhow.
maximuze danage, 6. distance closs, use clese-tnng sttacks redice opponent's health
siper gouge full, use Super
7. the distance far. you can eitter approach the opponent or use long ragge
soV Curzent state:
stnadee raluae their health.
You Ken Street Highter Your opponent Yarg.
K. You can use Megafineball the suprr bar gange is more than 3t
You
9. You can Super Altack (Shippus-Jin Junni- Kyskui ONLY your super count
from Opponeen
Remaining
You CANNOT
Attck (Shippun Jutrii Kyaion super
Health:
Your Health:
Sintepically choese
game your opponent get stunsed, tyy powerful moves mosimize damage.
Opponznn's Healt:: 134
Super Bar Grage:
the Valid action set tett
Your Super lar Urage: 52
Move Closer
Oppooeet's Super Bur Guage: 9
Mener Away
Count (Couns of activated super
Fireball
Your Count:
Magapunnc
Upponeen's Super
Hurnecane
Guage:
Megaffieball
Your Stun Guage:
Super
Opponent's Stun Bar Gunge: 10
Is 1sStunned (0. not stunnod, summedi
Mediun
Your Statie B
High Punch
Opponeet's Special Status: 0
Low Kick
Medrum Kick
Based above information, analyne the curnent cituation
HHihh Kick
you should for
step. Then. should oupput
Low Punditl ow Kick
actions you want
game
Medium Punca+Mediuut Kick
You should only respend actions
set.
Panch-Hiig Kick
You should only respord the fornst described below, and you should not output comments or other information.
Jump
Jump Away
Provide your requatee the strics format: sete Ressoning
ahe Dasision Chiipui Fumuat
Analyse the grovided game staze and deternino the *frost actions** to take nest.
Reture your decision in the following cxact Socaut:
abo Reasuning
<2 detailed summary why this action was chosen-
the Actions
consecutive actione the valid action set>
The Ressoning' Gold peovides a explanation of way actius
host choics
The the Accions field costain only most two of the valid octions.
Figure 8: Action inference prompt for reflection-planning agent playing Street Fighter III.
on the target task and the current game state. The user prompt includes: (1) the main goal of the game, (2) the previous subtask (generated by the recent planning module), (3) the last executed action, (4) self-reflection the last action (generated by the recent reflection module), (5) the current state, and (6) the expected output format for the subtask reasoning task.
Action inference prompt. As shown in Figure 8, the system prompt outlines strategic guidelines for playing Ken, a predefined set valid actions, and the required output format. The user prompt contains: (1) the current subtask (provided by the recent planning module), (2) the last executed action, (3) the corresponding self-reflection (generated by the recent reflection module), (4) the current state, and (5) the expected output format for the action inference task.
Reflection prompt. As illustrated in Figure 9, the system prompt provides detailed instructions for an agent to perform reflection. The agent is required to analyze whether the last action successful based on state transitions. The user prompt includes: (1) the target task, (2) the current subtask (generated by the recent planning module), (3) the last executed action, (4) the previous state, (5) the current state, and (6) the expected output format for the reflection task.
C.3 EVALUATION METRIC FOR STREET FIGHTER III
Single-Agent Play. In the player- mode, the agent faces a series of 10 stages against in in-game rule-based bots. The game ends either when the player loses stage or successfully clears all 10
21

---

Published as a conference paper at ICLR 2026
System prompt (Reflection)
You are helpfal Al assistant integrated with 'Street Fighter ID: 3rd Sinke' on PC, capable of andyying in game consexes and determinung whether as executed action los taken effect Your toak ovaluatc the success of serions bosal changes provide logical reasoning.
answer following questioss step by stop
reasoning based
oa the last action and the states.
What is the executed action and its desired resll??
What the difference betwern Provide reaeeringg
stries? Compare
action was sucocisfal)
the probable couse! Give
ouly one cause You should sammarize the reasening in a clear and concise munner, providing a logical explanation for the success failure of the last action.
User prompt (Reflection)
AeU Target takk Defeat the oppanent
playing Stroet Fighter Your opponent Yang
tite Current subtack
You are facing lef.
The current subtask to close the distance by using a forward dash or advancing
Distanee from Opponent close
special then apply with medium and heavy attacks soluce apponent' hoalth.
Time Remaining:
Your
not Last exeeated action.
Megatireball
Super Bar Guage:
Orpanen's Health. 123
Move Closer
Super Bar
Opponent's Super Guage:
A4U Previoms
Super Count (Count of activated super move]:
You are playing Ken in Street Fighter 3. Your opperient is Yang.
Your Super Counc: O
You are facing lett.
Opponent's Super Count:
Distance from Oppanent: close
Remaining:
Guage:
Heallth:
Orponent's Stun Rar Ghagge:
Your Hualth: 161
LsStinned (0: not stunned, I: stumed];:
Opponznt's Health: 134
Status:
Bar Guage:
Oypaneni's
Super Guage:
Opponent's Super Dar Guage:
You should only respond is the Fornat as described below. aud Self reflecion
Super Count (Count of activoted super movey:
Your Super Count:
Opponent's Super
Guage:
Your Stun Bar Guage:
Opponent's Stun Bar Guage: 10
PEA Self reflection summary
IsSturned (0: not stunned, stunnedy:
Opponen'ss Special
Figure 9: Reflection prompt for 'reflection-planning agent playing Street Fighter Il1.
stages. Therefore, the evaluation metric can be straightforwardly defined as
Score Number of stages cleared by the agent 10.
Multi-Agent Play. To evaluate models in competitive multi-agent environment, conduct pairwise matches between all agents and compute Elo ratings based on their win rates. For each pair, three games are played obtain a reliable estimate of relative performance. The resulting win rate matrix and Elo scores are presented in Figure 4(a).
We adopt Bradley-Terry model formulation (Bradley & Terry, 1952) for Elo estimation, where each model's rating is iteratively optimized using gradient ascent on the log-likelihood of observed outcomes. The gradient computed based on the expected win probabilities derived from current ratings, using the standard Elo transformation:
1
P(i beats j) = 1 10 R.0/400
where RG and R, denote the Elo ratings of agent i and agent respectively. The expected probability of defeating increases as the rating difference R, R, becomes larger. After optimization, we shift all Elo ratings so that their mean equals 1,500 for intuitive interpretation. If two models receive identical ratings, the one that won their head-to-head match is ranked higher.
C.4 EXPERIMENTAL CONFIGURATION FOR STREET FIGHTER III
For all open-source LLMs, including 1amma-3.2-1B333 2wen-225-3BB7B and Minitron-4B/SB. temperature of 0.0 and repetition penalty of 1.0. During L.L inference, pause the
22

---

Published as a conference paper at ICLR 2026
Models
SF3
Rank
Models
Ageat
SF3
Rank
Models
Input
SF3
Rank
Random Agent
10.0
12
Random Agent
10.0ga
9
Random Agent
10.0ga
10
Llamu-3.2-11
0.0gae
13
Zero-sbot
Llama-3.22 2-3B
Qwen-2.5-3B
2.5-7
20.0-cm
16.7a115
13,3
10.5
4.5
7.5
LJama-3B
Reflection
Ref-Plan
Planning
30.0
20.0gm
1.5
6
GPT-Ao
Image
Both
23/7am
2443
13.3gm
Minitron-4B
16.7
7.5
Zoro-shot
29, 29.7
3
Gemini
Imuge
16.7
GPT-4o-mini
Minitron-BB
GPT-4o
03-mini
33.3
29.7+143
23,3
16.7a115
7,5
2
3
1
GPT-4o
Reflection
Ref-Plan
Planning
23.3
30.0
2
4.5
4.5
Claude
Image
Both
Both
Text
23.3
16.7
20.0g
33.3
Gemini pro
13.3
10.5
Table 11: Ablation study for agen-
Claude-3.7
16.7
7.5
tic modules Street Fighter III.
Table 12: Comparison across
Deepseck-R1
20.0=co
4.5
modalities on Street Fighter III.
Table 10: Gameplay score on Street Fighter III.
Street Fighter III game environment. We set the maximum number of game steps to 10,000. Due to the in-game time constraints, episodes do not reach the maximum of 10,000 steps. A single stage (best-of-three matches) is usually resolved within 100 to 200 steps, rounds either timeout end early when one player's health reaches zero.
C.5 RESULT FOR STREET FIGHTER III
All reported results in Tables 10-13 are computed as the mean and standard deviation over five independent runs, using 'zero-shot' agent for each model configuration. To establish a baseline for comparison, we additionally included random agent that selects an arbitrary action uniformly at random. This agent was evaluated over 30 episodes, and its performance is summarized in tables.
Single-Agent Play. The performance across various LLMs was evaluated as presented in Table 10. The smallest model, Llama-3.2-11 completely failed to comprehend the current game context and consistently ignore the required output format, performing even worse than a random agent. On the other hand, all other evaluated models surpassed the random agent, demonstrating varying levels of competence. Commercial LLMS generally outperformed their open-source counterparts, with GPT-4o and o3-mini standing out as notable examples.
To investigate the effectiveness of agentic modules, we conducted an ablation study shown in Table 11. For Llama-3.2-3B, the Reflection agent showed notably superior performance, indicating that reflective reasoning significantly aids in aligning actions to game dynamics, possibly by allowing the model to reassess and correct previous outputs based on feedback. Conversely, for GPT-4o, both the zero-shot and Planning agents performed remarkably well. This may suggest GPT-4o's inherent capability for generalization (zero-shot) and structured sequential reasoning (planning), enabling efficient decision-making without iterative reflection.
Input modality may significantly influence agent performance, since spatial information in fighting games is expecially important for gameplay. Since our implementation simplify character distance into three sparse levels, we expected image inputs would enhance spatial understanding and consequently improve performance. However, as shown in Table 12, GPT-4o surprisingly demonstrated decreased performance when using image inputs, possibly due to limited visual comprehension capabilities. In contrast, Gemini and Claude effectively leveraged visual data, improving their gameplay scores.
Lastly, Table 13 demonstrates intra garr- generalization capabilities when fine-tuning models on Ken- specific gameplay data and evaluating on Chun-Li scenarios, which represent nut-of-distribution conditions due to differences in action spaces (as previously detailed in Section C. Remarkably, the previously cermat-incompiant pretrained Llama-3.2 3.2-1B learned to follow the required output format effectively and exhibited excellent gameplay performance after fine-tuning, even in Chun Li gameplay. A similar improvement was observed in Llama-3.2- 3.2-BB, which even surpassed GPT-4o in performance. These results highlight the significant impact of targeted fine-tuning demonstrating that relatively small-scale LLMs can achieve substantial gains in task- aisk-specific capability.
Multi-Agent Play.
We evaluate agent performances in a multi-agent environment by conducting pairwise matches among 8 I.LMs, all operating in zero-shot setting. The results provided in
23

---

World 3, Stage for the evaluation of Published as a conference paper at ICLR 2026
Model
Finetune
SF3
0.0+an
Llama-3.2-IB
42.0
Llama-3.2-3B
12.0+t1
40.0
GPT-4o
10.0
Table 13: Intra game generalization score of Street Fighter III.
Figure 4(a). Each pairwise matchup consisted of three independent games, with each game played in best-of-three format determine the winner. To ensure fair performance comparison, both agents used the same character-Ken-inn all matches.
Notably, unlike the single-agent evaluation results in Table 10, Minitron-8B consistently outperforms all other models in the multi-agent arena and achieves the highest Elo rating. This divergence raises the possibility that the involvement of other intelligent agents could alter the game dynamics, perhaps due to increased strategic diversity or emergent ad advesariial behavior.
D SUPER MARIO
D.1 GAME DESCRIPTION FOR SUPER MARIO
Environment. Super Mario (1985 Super Mario Bros) (Christian Kauten, 2018) is a side- scrolling game where the player controls Mario to avoid obstacles, defeat monsters, and reach the flag. In this environment, Mario progresses through the game using directional key controls (e.g., 'left' and 'right' keys) and jump actions. Mario should either destroy or traverse obstacles (e.g., bricks, stairs, pipes), avoid or defeat mon- sters (e.g., Goombas, Koopas) by jumping on them and avoid falling into pits. For implemen- tation, we adopt the gym_super_mm. lmr.mmriii mariobrros environment (Kauten, 2018), which is widely used in the reinforcement learning (RL) community. Specifically, environment, where Mario plays within 256 356x240-pixe screen with a black background, as shown in Figure 10(a). The environment consists of 8 worlds, each containing 4 stages. We use World 1, Stage 1 as our default evaluation setting (for Table 3). However, the environment supports evalua- tion across any world and stage. For example, intra-gane generalization (for Table 7).
Monster
Goonbu
(a) Screenshot.
(b) Object Patterns.
Figure 10: Screenshot and assets of Super Mario.
use the SuperMarioBror -vi
The environment only provides RGB image frames obser- vations. To convert visual game states into text input suitable for LLMs, we apply visual pattern matching parse the exact location of each object on the frame. As shown in Figure 10(b), in the SuperMar oBros-vl environment, the pixel-level visual patterns of objects remain stable across frames. By maintaining set of pixel templates for all game objects as game assets, we perform 2D visual pattern matching to parse the presence and exact location of each object in the scene. These parsed object locations are converted to 2D coordinates (z. y) and formatted as text, which is then passed to the LLM part of its observation input.
Observation-to-Teen Conversion.
Action Space. We constrain Mario to only move in the 'right' direction to simplify the control space. Mario can 'jump' at varying heights, and by constraining action spans game frames (i.e., frame skipping), Mario's 's jumping ability discretized into levels. Jump Level 0 corresponds to walking forward without jumping, while Jump Levels through 6 represent increasing jump heights, with Level 6 being the highest possible jump. At each game step, the LLM chooses a jump level from 0 to 6, determining the jump height Mario moves the right.
24

---

Published as a conference paper at ICLR 2026
System prompt
User prompt
ussstest pliping Super Macio gasse
You gei the
withest dying sooding decitiin obstacks lnveriies
Ofject Resoription
Bricke (90, 88) (1M, 95, 95)
-Ticke Bindodie Madee coeais enle, (Sor licte)
Quatton Tuocee
99), 1400 (174,150)
-queiies Blecke Reveal cokas posstcapo whet hit deadivase sha (Siec. H6dl))
Inactivated Blocks Nasz
-he ralling mostts loong a life
Moneer Useebbe None
Warp tiee Itassr above the groend, oo Mirio mes jeny over them when oppear fhom (Sinc Mocleigttll
Moneer Korpac Newe
Mocsaer Groorbe Bask cneriyy can defisned by jurging on (Stex. 16
Pl: start Noce, end Nore
Moonier suoppz stanic snars: retraaa into exii when parped thon: 20x24)
-aar Pipe: (221A15))
Mushroo
lager, gast protection
(6x10)
Touch
-Gnondt
gaeas y012.
(Note reter the left ctjoc)
Aatian Descriptonn
-Mtns (size: (0ci costinurally moves to tho right at find speal
You mas choosc an quarepraae juzto leve to respend to spoantng obeacks
Tach jamp level dermiiss bal
Bow for uaae busiontally distsod)
pemp heiger)
grosed
janp, wilk)
-Love 1:142
-Love 1::36:0
- -Lovel 2:-63 is --553 in
- Lovel 4.+70 is in:
Level 5: +77 is -65 i
-Love 184 in
key chociing
juna right muttonn
While Mario fillws "ponttlii
Mario be "Mackel otjoos ridf definee
asbome cuemaa"
MMric lop boiido Hocks, pipcs, stars
Game screemshot
At each ganeo stg, you will receive the eanart gasao store in the fellowizg fenaat.
Poriion Macio (x.))
Pourton ebrectic
MAR
000 100100
E 1X00
1-1
TIME
-Ricks: jel. yl y2)...
inactivated Bloda:
MMontrr Gooetas (pl.
-Monke Koepes: [id. p6).,
PP. vast (x).
(2)
-Wap Pear (jcl. height -.]
Jene Maslroore: [01.,10....
Blacks (xt. yTx .]
top-kett
objeet)
should fao
tophin applicatle: why chour jame level
kamp Level a (whare is on inege don to ialketing the closes jump leed)
You MUST only remoend is the forma with the mela. WM Actinnia as below: Actions
heghare...
lapp Level
Figure 1I: Action inference prompt for 'zero-shot' agent playing Super Mario.
D.2 GAMEPLAY PROMPT FOR SUPER MARIO
Figure 11 shows the action inference prompt used by the 'zero-shot' agent for playing Super Mario. The system prompt contains most of the gameplay-specifie knowledge. It includes (1) the main goal of the game, (2) detailed descriptions and sizes of each object, (3) explanations and safety notes for each action, and (4) the expected input-output format between the LLM and the environment. The user prompt provides the current game state as list of all objects detected in the frame, represented by their top-left corner (x, y) coordinates obtained by visual pattern matching. Given this prompt, the LLM agent infers the appropriate jump level to advance safely toward the flag by avoiding obstacles and monsters.
D.3 EVALUATION METRIC FOR SUPER MARIO
The goal of Super Mario is to reach the flag located at the right end of the stage. Since the gym super mario bros environment provides Mario's current position on the map, we define the evaluation metric as the proportion of the distance traversed toward the flag before Mario dies. Formally, the normalized score is defined as:
Score = dist(xMarioon Fstart) F (zst flagg Wstart) x 100,
where I' Marios I flag- and X sstart are the x coordinate of Mario traversed before die, that of the flag, and that of the starting position on the map, respectively.
25

---

Published as a conference paper at ICLR 2026
Models
MarerMario
Rank
Models
Agent
SuperMario
Rank
Models
Input
SuperMario
Rank
Llama-3.2-11
18,7
12
Zoro-shot
21.2
34.1
Qwen-2.5-38
L.m3e-3.2-38
23 23.4
31.8
11
Llama-3B
Redlection
Planning
32.4tsa
GPT-4a
Image
Both
27.1
27.1gm
6.5
Qwen-2.5-78
27.2:
9
Ref-Plan
31.8g
Minitron-1B
24.4asa
Text
38.0
2
Minitron-8B
31.3
29.619a
Gemini
Image
28.5
5
Go--io-mini
28.8
GPT
Rellection
Planning
29.4-125
Both
40.9196
o3-mini
GPT-4o
34.9
34. 2217
Plan
Claude
Imuge
Text
25.6164
28. 13
4
Gomini-2.5ppro
38.0
Both
22.6am
9
Chande-3.7
31.7
Deepseek-R
28.7g0
Table 15: Ablation study for agen- tic modules on Super Mario.
Table 16: Comparison across modalities on Super Mario.
Table 14: Gameplay score on Super Mario.
D.4 EXPERIMENTAL CONFIGURATION FOR SUPER MARIO
For all open-source L.Ms,, including Llama-3.2-11 3.2/-3..-1B/BBB, Owen-2.5-33B775 and Minitron-4B/SBh we use temperature of and repetition penalty of 1. During LLM inference, pause the Super Mario game environment. We set the maximum number of game steps to 100. run experiments with 20 trials and report the average score with the standard deviation.
D.5 RESULT FOR SUPER MARIO
As shown in Table 14, Gemini-2.55 2-5-po achieves the highest score of 38.0 on Super Mario, followed by 03-mini with the score of 34.9, Among open-source LLMs, Llama-3.2-3B and Minitron-8B perform competitively, achieving scores of 31.8 and 31.3 respectively, which are comparable to the 31.7 score of Clludd-3.7-soomee fail to correctly estimate the parabolic jump trajectory when Mario is mid-air. This change of moves often results in Mario colliding with obstacles or being killed by monsters.
Qualitatively, most LLMs, including Gemini-2.5 2.5-pi- frequently
Table 15 shows the effect of reflection and planning modules gameplay performance. Among these, the reflection module has a more pronounced impact. Specifically, when Mario is stuck in front of high obstacles such as warp pipes, the reflection module enables the agent revise its previous low jump level decisions and select a higher jump level, allowing it to overcome the obstacle and proceed, thereby improving the final score. GPT-4o achieves the best score of 34.1 when both reflection and planning modules are used. In contrast, module is used with score of 32.4, indicating that Llama-3.22 3.2-3B may not benefit from the additional planning module or be easily disturbed by its response.
3.ma--3223B performs best when only the reflection
As shown in Table 16, using image-only input observations consistently underperforms compared to text-only inputs across all models, including GPT-4o, Gemini-2.5-pra and Claude-3.7 This suggests that relying solely visual input makes it more challenging for models extract detailed information from the game scene or to perform spatial reasoning, i.e., estimating distances between objects. In contrast, when both text and image inputs are provided, GPT-4o and S.aude-..7-somm show performance drop. while Gemini-2.. 2.5 pro shows improved performance. This indicates that multimodal input can be beneficial when the model effectively integrates complementary information from both modalities.
E ACE ATTORNEY
E GAME DESCRIPTION FOR ACE ATTORNEY
Environment. Ace Attorney (Capeom, 2001) a court- room adventure game where players act defense attor- neys, gather evidence, and cross-examine witnesses. We target the first episode of Phoenix Wright: Ace Attorney Trilogy on Steam (see Figure 12). We define four subtasks: threc-question multiple-choice quiz where the player selects the correct answer, and three cross- cross-examination tasks where the player presses witnesses for more details
Episode The Fisd Turasbat
Figure 12: Screenshot of Episode 1: The First Turnabout.
26

---

Published as a conference paper at ICLR 2026
System prompt
User prompt
You mo in All defena attmmay in an intreatiive Acc Anumoy-ttyl taall The gamo advanoos screenhy- sences huacalun your choices, and your godl is win by managing dialegre and coilener elTeriivlyy "ONLY" perfocts actions permitted by the currenty visible screen.
Cxmer State
|Recest Convenationl
[3025 06-02 17:18.20 Alia; Yos Bi., you Hasor?
Respensisiitiis:
Choose the best nuttipee
Moniuor dialggue for enss neview cvidknes proflex
Brave Open the Count Record sealor 1i9000 sipler,
|*e Corveraaion
vettinory!
datect contruiciion and present evidence.
Last Coutt Recond
eng se theck Time*"; 2015-41--15 12:44:13
Camoplay Cuidclince
"CCast Roood londmnett
Press continee dislogue and "Tabl to access the Record.
1: Anumey's Balge- Noore would beliese was defane otteney if! didei eary this.
""ONLYY access
Rocord absolutely necesury:
the
Tame" sigtifronlly outdated
cunest dislogue
actione moet be based willl on invertonn dislogue The mn-eroan dillegus defined the very jast eatry is the "Curent Staress Recest Conversttios] which is masked The Conversaion Corrently on Screes"")
5. Blackent Recaed Eleexisity to Ms. Foctret's buildag was from ofe
soon
There are owo types of inpoeiant dislogue: (1) repular dialgge ii no cole fermating) and (2)
Prodicee.
rostmmny for Craae Focmiinrionn deplayet in green lolloe-aeeitttt
1: Bi good deforne Chief
Co boss, and
The tinal godl the game to stencity contrddctioss between the on screen restitiony and the Court
Record, and to presen: evidence proving Shat hiss testumony being stown.
5. Echo (ag, 36) Discoaree Ms. Fottools body, Newapaper salomran who saw Datra flas the scone.
"IMPORTANT (Cros Examination Eligbllity.. +* You may perforn Cross- Examination sctions ("Pros the momert unlimmay" "Proent the sclotto exidenee)) when buth corditions below
Lad Decisiem: Nosr
The jaaimony dispaysd colce-2001000
State" includes *Cross-
Exumization!*
"Pesible Optizea" (Jcciive Typeat:
The to contradicioon you
The Comcenationn Curnntly vaiimmny susgended confimed.
ciarly relates
1. Ok
"IMORRTANT (Action Spategy):** Whes bol Choss Exunination Elighility conditions are satiefied, use wither the follwing two Drows- Fromirorion schioe If your addttinnl
the fillwwing fareet:
darffcctionn
Reasoning
press "Press the moment of testinon"" (represazteed by "Hold it" is [Recent Conresations]:: Horvever. if you are cortider and ready to expose filse tectmooy. wait uniil the cortrdictog on screen costimatty apperes then pros "Ta"" sdea dez apprppiate eviltroe, and exceute "Presen: the selested exidence" ceproserted by "Ojjettin"
[Your happ-----eee rensmii base))
wel Actions
[Raoe Comeration:).
sorrect optios Seoe dae "PPosslle Oycion!"))
|*RUNILy" y curput the *INITIGHR** nmmberorrounniing
+100 NOT use these octors for morely suspicious anbiguous discrepancies Trigger them only when there is defintivve contidiction such when the testimony cirectly and logically conflicss with the actual rosand
Game screenshol
+000 NOT report that alrody recoded in ala Decisiond the
rodimany
Only
your atalysia (h* Conversator Currenlly Scceen"""
memory indianso contrudation, cominue presing uniil
comospanding tosimoty appors
prosiass taaimuny deplay
proo key
Additional
Consunntly asucas the dislogue for cues and adaps your strsegy is evidence smerge.
Remember every piece of teetimony contiins commadiction only mitiate coee-comiiteionn
when thece is clear and destiitive evidence of incorsinency.
lbxd it right there!
**IMPOR I:AN Only select an action from the candidone list by responding solely with the "INTEGER** * number conopprndnn to the selested optios.
Figure 13: Action inference prompt for 'zero-shot' agent playing Ace Attorney.
presents evidence to expose contradictions. We use
Harmony (Pardeike, 2025) with a BepInEx plugin (Contributors, 2025a) to hook the game's 's source code launch, capturing states such as dialogue text, arrow-button visibility, keyboard inputs, and Court Record entries. The hooks save states as txt or json files and monitor a command tx file for inputs, injecting them into the game in real time.
Observation-to- Text Conversion. All states remain in text form and require only minimal post- processing. We map speaker indices to character names (e.g., index '2' - 'Phoenix Wright') using predefined mapping and replace original names with arbitrary aliases to prevent contamination (e.g., 'Phoenix Wright + 'Alias'). We also convert Court Records and multiple-choice candidate orignally stored in -jsonninn continuous descriptive text when they exist.
Action Space. The basic actions include pressing 'Ok' to progress the dialogue and 'Tab' to access the Court Records. For multiple-choice questions, the action space expands to include candidate options. During cross-examinations additional actions become available: 'Left' to return to the previous dialogue, 'Press" to question the witness further, selecting relevant evidence from the Court Records, and 'Present' to introduce during the examination. Each option carries an index, and the model returns only the corresponding integer.
E.2 GAMEPLAY PROMPT FOR ACE ATTORNEY
Figure 13 presents the zero-shot agent's action-inference prompt for Ace Attorney. The system prompt specifies (1) the game's goal, (2) procedures for dialogue, multiple-choie questions, and cross-examinations, (3) rules for accessing Court Records and presenting evidence, and (4) the expected I/O format between the LLM and the environment. The user prompt lists recent conver-
27

---

Published as a conference paper at ICLR 2026
AceAttomey
Models
Agent
AceAttomnyy
Rank
Models
Inpui
AceAttomey
Rank
Llama-3.2- 3.2-1B
1.3
12
Zero-shox
5.7
85.3
L.ama-3.2.3B
Qwen-2.5-38
4.6s1
Llama 30
Reflection
Planning
4.6
4.6s12
6,5
GPT-4a
Both
53.5g10
5
Qwca-2.5-78
9,3102
10
Ref-Plan
3.8
Gemini
Text
58.7
Minitron-48
35.7a43
Both
52,610x
6
Minitron-8B
29.9
Zero-shox
49.9
GPI-4o-mini
28.4g
GPT-4p
Reflection
Planning
52.7ans
85,3
Claude
Text
Both
71.3am
81.9sia
2
GPT-4o
85.5
Ref-Plan
52.8gas
Gemini-2.5-pra
55.7
Table 19: Comparison across Table 18: Ablation study for medd--iiss on Ace Attorney. tic modules on Ace Attorney.
Claude-3.7
81.9ata
Deepseok-R1
83.3g1
Table 17: Gameplay score on Ace Attorney.
sations highlightin the conversation currently visible on the screen and provides timestamped Court Record entries. Using this prompt, the agent detects contradictions, selects the correct actions, and manages dialogue and evidence to advance the trial.
E.3 EVALUATION METRIC FOR ACE ATTORNEY
Each subtask begins and ends at fixed points, with screenshots at both start and end and the preceding conversation history provided, and yields reward and step count f.. In the multiple-choice task (MC), r, is the number of correct answers out of three, while in cross-examination tasks CE1, CE2, and CE3, is for pass and for fail; any failure incurs maximum step count of 50. We normalize each reward and step count against fixed benchmarks F, and &,, namely (3, 25) for MC, (1, 11) for CE1, (1,3) for CE2, and (I, 4) for CE3, by computing
P.
8
=
7
t
and assign each task a difficulty weight wi E (1,4,2,3) reflecting MC, CEI, CE2, and CE3 respectively. These weights proportional to each task's difficulty and the minimum number of steps required (the benchmarks). We then form weighted averages
Σ W.Ps Σ, Wi
Σ wi Σ
A
B
Wi
and combine them into composite score
Score 100(a A + (1 x) B),
with a = 0.7 (70% for accuracy, 30% for efficiency). All experiments were repeated three times, and report the sample mean and standard deviation of score.
E.4 EXPERIMENTAL CONFIGURATION FOR ACE ATTORNEY
For all six open-source LLMs 1lamm-3..2-1B33B. Qwen-2.5-3B/7B, and Minitron-4B/SBB) we set the temperature to 0.7, apply repetition penalty of 1, cap game steps 50, and limit conversation history to the most recent 20 exchanges.
E.5 RESULT FOR ACE ATTORNEY
As shown in Table 17, o3-mini tops the leaderboard with score of 91.7, followed by GPT-4o with score of 85.3, Deepseek-R] with a score of 83.3, and Slaude-3.7-sommee with a score of 81.9. Among open-source models, Gemini-2.5-pro and Minitron-4B score 55.7 and 35.7, respectively, while smaller Llama and Qwen variants remain below 20. Notably, Qwen-2.5-3B completes the first cross cross-examination in one out of three trials, where other open-source models struggle. Models scoring around 30 often misinterpret key details and repeat the same mistakes until they reach the maximum number of steps. Models below 20 sometimes reason correctly but mostly select irrelevant options, whereas the lowest performers fail to follow the required format and cannot advance further.
Table 18 compares our reflection and planning modules. The base Llama-3.2-33 model scores 5.7; adding reflection planning alone yields approximately 4.6, and combining both drops the
28

---

Published as a conference paper at ICLR 2026
score 3.8. Smaller models like Llama-3.2-3B often focus incorrect details and produce flawed reasoning when reflection or planning is faulty. GPT-4o scores highest with reflection alone (85.3), since reflection prevents repeated errors and helps spot contradictions. Planning alone adds little, and combining it with reflection lowers the score to approximately 52.7, as it attempts to resolve contradictions not visible on-screen.
Table 19 compares input modalities. GPT-4o and Gemini-2.5-pre score 85.3 vs. 53.5 and 55.7 vs. 52.6 for text-only vs. multimodal imputs, respectively, while Claude-3.7-sonne falls from 81.9 71.3. Because we supply complete text descriptions of every on-screen element and its context, adding visual input brings no improvement mmproemmntt-ooffmingg that text alone suffices to play Ace Attorney.
F HER STORY
F.1 GAME DESCRIPTION FOR HER STORY
Environment. Her Story (Barlow, 2015) is interactive adventure game where players explore police interview clips to uncover a hidden truth. The player begins the game by accessing an old desktop interface, where a program called A.G.O.G.IIC keywords into the database to retrieve up to five video clips whose transcripts contain the searched word. By watching these clips and gathering clues, the player repeatedly formulates new queries to reconstruct the underlying story. To interface the game with our code, we use Harmony (Pardeike, 2025) together with Unity Doorstop (Contributors, 2025b) to log internal game states to a txt file. Specifically, each line in the file is a JSON object representing a snapshot of the game's 's state given moment. Fach object contains an event type and associated metadata. The following examples illustrate key elements of the logged game state. For clarity, we omit auxiliary metadata that are present in the actual logs but are only used for debugging or UI-related purposes, such as video IDs, screen resolution, and UI element positions.
Database is open. The player can enter
Load title screen: ("status": "title")
Load T.,OGG. Database: ("status": "start game")
Query: ("status": "query", "keyword": keyword)
Get query result: ("status": "query result", "num total": number of clips containing the keyword, "num_visible"" number of clips shown, "video": list of ("new": I if not viewed, otherwise 0, "session": recording date, "outfit": visual description of the thumbnail)
Open video panel: ("status": "open_detai"
Close video panel: ("status": "close "close_detail")
Play video: ("status": "play-video", "script": transcript of the video}
Close video: ("status": "elose_video"
Observation-to-Tess Conversion. We aggregate the game states and convert them into textual observations. Each observation includes summary information such as the number of clips containing the keyword and the number of clips shown. It also contains per-clip metadata, including the recording date, thumbnail description, viewing status, and the transcript if the clip was viewed after the query.
Action Space. The original Her Story game is designed as a point-and-click interface, where the player interacts with the game by typing keywords into a search bar, clicking on retrieved clip thumbnails, navigating panels, and controlling playback. These interactions rely on low-level input mechanisms such as mouse movements and keyboard input. To reduce complexity, we abstract these low-level interactions into two high-level actions: searching with a keyword and playing the retrieved video clip.
Search [keyword]: Returns all video clips whose transcripts contain the exact word. It consists of three low-level GUI actions: (1) clicking the search bar, (2) typing the keyword, and (3) pressing Enter submit the query.
P Play Video [i]: Plays the i-th video from the current search result list. It consists of four low-level GUI actions: (1) clicking the thumbnail of the video clip to open the panel, (2) clicking the thumbnail within the panel to start playback, (3) either waiting until the video finishes or
pressing the ESC key exit playback, and (4) clicking the Exit button close the panel.
29

---

Published as a conference paper at ICLR 2026
F.2 GAMEPLAY PROMPT FOR HER STORY
Figure 14 shows the action inference prompts used by the '7ero-shot' agent play Her Story. The system prompt provides instructions covering: (1) the main goal of the game, (2) the type of information each video clip may contain, (3) behavioral rules the agent should follow-such as avoiding repeated keywords in the search history- and (4) the expected input-output format between the LLM and the environment. The user prompt includes: (1) the last executed action, (2) the current game state, and (3) the search history.
System prompt
User prompt
You traellger agret playnng sidee-baerd my evry game..
excratriactton
Your poil unsever
sourlias posiilll by thinking logically stroegioolly.
Youacarched
"nimar". found
ovacly "Mus""* followias
LIMITDD FIRST ENTRIES..
SSorc (eyyoodd) Revens all vidoo where the conasine the most wond Search murter
Sonch naulte
Video (j) Plae the vidzo from scarch nosth Pay Video
[1998-06-11] Thurnnaall lin jadke, her diwn Statue Viewed- Saigt
Each video containe
1. [1996.06 06-1)) Thunbeall Blue jocket, lair down Stats Viewed Scrit:
Rocoeding date
"haarbeall-a sased descrisee of the video
2 (1996-06-25) Tharicaail Saurt-lloved ponter biee blesse aac ip Binee on side feo SSomae Virwod Serpe soent prozet vo much one thet aegemente bee boon simmeriy fora
Viinxing cans (Vieooa 'Ne Virwad)
Scrpp (If you have not vircod video, be giver (*.)
engloved glass pracet was
nice mime pa peisoss would have in story.
only action
specally
coudts
[1904 06-27)--2
Lorgodeaved
visibe, MUST
light..
You MUST NOT play Vidio or higher For exsmple, Video Video INVALII
somrwhire Upissin trinnk Hlaeen' lockid cinss"
of
earcs nourned romals, then theme ard NO vidzos acsilante play. MUST chooee
[1994-28-30] Thammnnall l'kmal blouur, Stanae Not
now, unsearched word issted.
viewed Stritt
You MUST ineeree the search Hinocy ccueatly
Thee scarch histry is littd from dlacn to newerr with the namber of search roults parcethono.
For currple, munkkr (4), wazad (IV mose you previoualy scarched munder and par soatts and abo soachee 'ibarr which rummred meals.
dM Search hitcey
mandr e4): Since (6)) Fimet (1): Fluerah (189, Pixe (?)) idnnity chose (it) alii (1) (1)or(11)
SUOULLD reglay a thet alrrad morsed "Viowed" tox urouiled veript Repliring sarse video ALWAYS ioturn osee sane script. dialagoe oolio, infociaatinn
eevealed
keyword
traneriptss,
verieiice will appear.
Game screenshot
At Your Goal.
Piegress the stary by watching new videos.
Use the vadoo content to iniie kaysonds for future warcha.
AM Outpor Formot
Abwaye respord tuding **burs** Silkwwing sactire:
rensaning.)
Al Artios
[Eaady actinn cide: 'Search [Leyword]] Play Video (J], No oiae teat]
IMPORTANT: You MUST super both wis koaceingg and WAN Actiona' in every engle requanes. either
sectoo mising your ouput wal be comiddreed invalid.
Figure 14: Action inference prompt for 'zero-shot' agent playing Her Story.
F.3 EVALUATION METRIC FOR HER STORY
To uncover the truth behind the case, the player actively explores the video archive by issuing queries and watching clips. We evaluate this behavior by counting how many unique clips the player has viewed from the full archive of 271 clips. This aligns with the original game design, where unlocking achievements and reaching the ending depend on the number of clips watched. Specifically, we define the score as:
Number of distinct video clips viewed
Score
x 100.
271
F.4 EXPERIMENTAL CONFIGURATION FOR HER STORY
For all LLMs, we use a temperature of 0.3 and repetition penalty of 1. We limit the maximum number of interactions with the game environment to 400 steps. We run each experiment three times and report the average score along with the standard deviation.
30

---

Published as a conference paper at ICLR 2026
Models
HerSSory
Rank
Models
Agent
HerStory
Rank
Models
Input
HerStory
Rank
Llana 3.2-1B
11
Zero-shot
4.2-1
Text
64.2a50
3
Llama-3.2-3B
Qwen-2.5.30
4,2
12
10
Llama-3B
Reflection
Planning
4,4-18
5.2g10
6
GPT-4o
Both
40.6=29
6
Qwen-2255-78
8.5gr0
7
Ref-Plan
5.4044
Gemini
Text
67.5g13
1
Minitzon
Both
64.9
Minitron-B8 8B
8.2g1s
8
Zero-shot
64.2
GPT-to-mini
21.2+3a
6
GPT-40
Reflection
Planning
61,5+00
59.4257
Claude
Both
Text
63.6gM
62.9e1a
GPT--4
64.2
4
Ref-
62.0
03-mini
66.5234
3
Semini-2.5-pp
67,5
Table 22:
Comparison across
Claude 3.7
62.9
5
Deepseek-RI
67.2
2
Table 21: Ablation study for agen- modalities on Her Story.
tic modules Her Story.
Table 20: Gameplay score on Her Story.
F.5 RESULT FOR HER STORY
Table 20 summarizes the gameplay scores of different LLMs Her Story. Commercial LLMs outperform open-source models in this task. Gemini-2.5-pe achieves the highest score of 67.3, followed by Deepseek-RI (66.9), 03-mini (66.3), GPT-4o (64.0), Claude- 3.7oonet (62.6), and GPT-4o-mini (21.1). The low score of GPT-4o-mini is due to its repeated search of keywords such as alibi, evidence and witness, even after those queries have already been issued. This redundancy reduces its ability to discover new clips. Among open-source LLMs, all models score below 10. Most exhibited similar failure patterns: they fail play unseen clips, repeatedly issue the same keywords, or search using entire sentences ferg..for the name "Simon", for keywords related to Luna's) 's) rather than meaningful single words.
Table 21 reports the effect of incorporating reflection and planning modules on gameplay perfor- mance. For Llama-3.2-3B, the 'reflection-planiing agent achieves the best performance, followed by 'planning'. 'reflection', and 'zero-shot' agents. This suggests that such modules can improve performance for weaker base models. In contrast, for GPT-4o, the 'zero-shot' agent achieves the highest score, and adding reflection planning modules does not lead to further improvements. These modules tend to produce information that may redundant for stronger models like GPT-4o, which can already generate effective queries without additional reasoning support.
Table 22 compares gameplay performance across different input modalities. In our multimodal setup, the visual input corresponds to the main screen of the L.O.G.I.C. C. abase interface, as shown Figure 14. For GPT-4o and Gemini-2.5-pro, using both text and image inputs underperforms compared to the text-only setting. In particular, for GPT-4o, performance varies significantly depend- ing on the random seed when using multimodal inputs. We observe issues such as repeated use of the same keywords and the inclusion of quotation marks around keywords (e.g.. "murders "), which often lead to failed queries. In contrast, Claude-3.77son 7-sonettt shows slightly better performance in the multimodal setting. We speculate two reasons why the visual input fails to improve performance in most cases: (1) most of the useful visual elements (e.g., clip lists, thumbnails) are already represented in the text observation, and (2) additional information that visual input could provide-suce such as the interviewee's 's gestures or expressions is only available when the video is actually played, and thus not present in the static visual input used in our setup.
G POKÉMON RED
G.1 GAME DESCRIPTION FOR POKÉMON RED
Environment. Pokémon Red (Game Freak, 1996) is role-playing game where the player navigates the Kanto region to catch and train creatures called Pokémon, battle other trainers, and ultimately defeat the Elite Four and the Champion. The player explores various environments, including towns, routes, caves, and buildings, encountering wild Pokémon and other characters. The gameplay loop involves exploring these areas, engaging in bumed battles with Pokémon, and managing team of up to six Pokémon. For implementation, we utilize the PyBoy (Backalfen) emulator to run the game. Specificcally, evaluation focuses segment where the player starts in Pallet Town
31

---

Published as a conference paper at ICLR 2026
and progresses towards Viridian City, encountering wild Pokémon and trainers. The game screen resolution is x60x 144 pixels.
Observation-to-Tess Text Conversion. Instead of relying on visual pattern matching, directly access the game's 's internal memory via the PyBoy emulator to extract relevant game state information. This includes detailed map information, the player's current coordinates, information about the player's party (e.g., Pokémon, their HP), encountered opponent Pokémon information, and the player's inventory of items. This rich set of information is then formatted text serve the observation input for the LLM.
Action Space. The fundamental action space consists of the Game Boy buttons: 'up", 'down', 'left', "right', 'a', 'b', 'start', and 'select'. Additionally, we define a set of higher-level tools to facilitate more complex interactions:
move_to (x, y): Finds and executes path the specified map coordinates (1r, y)- Interact.with.objer (object.name)) Interacts with a specified object in the environ- ment.
wazp_wi wwt.wwittwwwpppoo.c (wazp.point.coord) Uses specified warp point to move to different location.
overworid wew.wordd.aa..rrasstt.... (direction): Transitions to an adjacent map in the given direction.
cont nue dialog (): Advances the current dialogue.
select.move.ing battle (move.name): Selects and uses a specific move in a Pokémon battle.
switch wwrttch.pkmn... wkkm_iinnbl battle (pkmn)name): n Switches different Pokémon the player's party during battle.
run_away (): Attempts flee from wild Pokémon battle.
inreiiemmii..bate... (it (item_name) Uses a specified item during a Pokémon battle.
At each step, the LLM can choose up to five consecutive fundamental actions or invoke one of provided tools.
G.2 GAMEPLAY PROMPT FOR POKÉMON RED
Figure 15 shows the system prompt used by the agent for playing Pokémon Red, which provides the LLM with the necessary game rules, action space details (including basic controls and available tools), information about different game states, and the expected input/output format. The system prompt guides the LLM on how to interpret the game state and decide on the next action or tool to to achieve the overarching goals of becoming the Champion and completing the Pokédex.
Figure 16 illustrates an example of the user prompt provided to the LLM. This includes the recent history of actions and their outcomes, the current game state (map information, player position, inventory, party, screen text, etc.), any recent critique on the agent's 's actions, the current sub-task (if any), and relevant memory entries. Based on this information, the LLM infers the next action or tool to use, following the guidelines set in the system prompt.
G.3 EVALUATION METRIC FOR POKÉMON RED
The goal in our defined segment of Pokémon Red is to progress through series of key storyline milestones. We define 12 predefined storyline flags, and our evaluation metric is the percentage of these flags achieved by the agent. The 12 flags are: Exit Red's House, Encounter Professor Oak, Choose a starter Pokémon, Finish the first battle with the Rival, Arrive in Viridian City, Receive Oak's parcel, Deliver Oak's parcel Professor Oak, Obtain the Town Map, Purchase Poké Ball, Catch new Pokémon, Arriv Pewter City, Defeat Pewter Gym Leader Brock. The final score is calculated as the percentage of these 12 flags that have been successfully triggered within given episode or evaluation period. Formally,
Number of flags achieved,
Score
x 100.
12
32

---

Published as a conference paper at ICLR 2026
System prompt
ur Active hicrewe Paloezs
Coro Rutes Ressiden:
Mus Goeb: Bacese Churpiio complox Nacdes.
Coessis A-Cartemmeacn tearadt B-Cresillak Stam-Mana O-fainMers Use ramnl trannnng cover
"Tate cuty
allowed Sexe: 10N TILLE, NNEW GAME. DOI'T qUiTt
"Picee Mov,, anerac. mona tue uncinnaacice tedo.
Piieiee soveaing tikes uakas axer and NrCoo preasssies paax lisoeves imparaat objocta wapp paint disceoodd, conide pniggiii
"Daaig"" Advvaso costine dakn Chasca D-Dadimnce
(castima) loploniaune
If 1-d--ad earopamtvv with adeatian bet: pron B abstteddall
oxpoltinng siniinng daklgg poss reporddyy mom
Ward sosnad
Deme oftica adi foom dillg prggulpeggrooce oneersoeee.
medommesee, nome oxtec ras) Traieer eg:
moniag
Map: (put Kay! grd ogee, down EuEiepp lett 'Phesabl Ofjecss" creenes
Worpharr ten Surty Brlliatt NC "Car ". "TatTT SPRITU, SIEN. Lodger (1). (.// Linoilaatle. lskmscable al-- RITICALL Takki SPRITE: SOON.
Walbaviry (CRUTICALL
-eneva Striagg
Prieties gelinig (MPCs sign, awvellng (llok roworee mos perggeenn (heal, lash, shaadeddoring, pod ieaceress Use nonayylliigg fien. se3. ym4), warg..tt perg
yy May Tranition
rngiee.
Invenoorf wwtpll___errrereeeeeee walkaaie 'ovewwrl rage).
Naszall Buass Refreace
upporiilyy
Start Opavyiiee main (Fidd andh
D--hat Marc chanskrrieent
apticable
Rovdda tll creand Ulage sue solieuse is deark deanty
iols, doll Mosz WALKABLE
7.30
poir
with
Movas "oarp
(incldee
Usage: sue sellkary vith sarp sorts.0 dunk dea-t
"inuctee: tent' wash Tworteas
Usage: uue selipraneatd rup tomuttos (drectoo toaty"
beanday
(Thee Olyenst "NMMiry
inove.oi dolig fill
Dagge
ane 2 Davig Saw Tools
contree clakgg
eptions Tysid Adsonc di (A/19)
ane Furta Save Taok
-dlect moor, burtianzss, purely Selec raass rame Getive Pobciookk now, UPPERCASI)
Usag:: aodiecke usone taar saxe "MOVE"Y
Usgg: awdioiddd plas
(pekcuan "HHM. NAME "AME"
uan Jame, betkiiinn, mane polccar, Janw (haad Rerr ratt' Cioss Segi seuptiond polcrssrjane" (Jou Curesl Party.
Usgg: juline
(ienc TTEM" polcmmrr Juan. NANS"")
Fee
Usge: www.wiilaaeeeeeeo
"Noee (Tabust)
Cameritarettes: Map, Plaver, Olpects
Sceen (ladeda: type), (Always provdad)
Necastatique (0pth wachhck aties.
adtai t0pi:
(cal
CORE LOGIC (Be Crecao)
lob Sabruk
edeak' "Neay' Befte iereddor and lest contrnirspiiiee ie "trfered" rave, explov "tntened cortine diving))
Saa Chedi Veuntty Commnttiaationn
Teo that That so fe uie caeak for ANAIl TOOIS. Chick precedaionuee teg "noce ss wallbbitty butta aod c1o9)
slss exploaation emeegh perfee frowr Tirge WALSAABL rusomotinn Low Leve Commee Str Puu ONLY irnemee
rame
harty: Deplain cheice cross, eabaak, mup, milec lonee si and meras, why reg. adporet bege, unong san, valld pario. Lontaand, wit Leawen, carol Eored Sictall lovees bast caago, oilige, exprevalll).
Ougus qall
pei ediecee..
RESPONSE FORMAT (Senet Atheeceo Rogiood)
go alreeee
and Lowarn burnet
"Lavce c.. The: reove utX rveaaedd Pale: 7
(sas sxedoe thcaal koma.. besposaltios)
ANd Actin.
Sobaak (Ponnded Next sabaa aooi ur "aieeee fyor inkene adtail?)
hadiiaatee: (Whe telleggonch plate sara wabaak miol? iasectt te formorojatol vio? Liviled,, sty?]
Fue: use sont mood samas caggery dipe acrtoses
Rodeetsey flo
Acties
-Ine Wealll sow Mestl (MAXX
OR
Conor confive
AIWAYS de
NOtth
Aebeww strn sed velooriie vildity
snadio Arhhe city Serear
Figure 15: Action inference system prompt for 'zero-shot' agent playing Pokémon Red.
33

---

Published as a conference paper at ICLR 2026
prompt
Recei Hoaury:
Summory: The plaer saccesffillyy moved from posttion (3. 6) (3, 2) within Rod's House 2F An intemntion was completed with an object named
"Talk "TalkTaPPin ToPonBooSSNSSTo "The player unterppol move
puint (7. but intially unuerzoolil due neel use spealii tol. Sabeegunlly transition from Reds House Red's House posttion
plaer successfully used the
Segtence: (oldestfise, (ieSuneess Focdback) (True, Surassfflly Intenet TalkToPiin BoSSNESTed "yree tnoljooe (x. dlett-,, y. dest- l)jersse jookmowe Nn, dest- y dest-1))
(onject mmme- MokkTooriinedNNSSSeeer
(teSnecees, Feeebaec.e (erdbacio)Falee "The destitation Warphhinnf Use lwarp. with warp, point tool "worrss_eoooooo wwapp/orr._wwiiiiii with worp_ point, (x (r_dest",, 1esttt-1))
buse warp with point, desenT.y v. dest-11) (lsSoseess, Feedbacii
Rf
Redaliouse2
State:
[Filtered Sereen Text)
N/A
[Selection
[Unemy Poketonn
Not buttle
[Curnent Party]
more Pokemons
[Radge Lad] Nix
[Current Money]: 83000
[Map Infa]
Map Name: RodsHlouself, (s_max my_mak
Map ypp
Expansion dieccton:0
Your positin ix, yJ: 1)
Your dacing direction: right
Action instuctionn
sin y-13
down y)
loft (5.g) x)
right: (x. x)-> (x)).y)
(Full Map!
(5-0)
(x-ti 01234567 (x-))
17500OW
2 PANDD00O
3 #700000
MYXXSOO
PINNDDO
(y-7)
[Notable Obrects]
SIGN REDSHOUSEIE TV
SPRITE MOM
Rocent Crilgne:
None
Next Subtasi:
Relevant Memory Entries:
1: Losom Fact Succesafully movel (3, in "RohhHoow"""
Figure 16: Action inference user prompt for 'zero-shot' agent playing Pokémon Red.
G.4 EXPERIMENTAL CONFIGURATION FOR POKÉMON RED
We configure all six open-source LLMs 1lama-3.2-1B/3BB Qwen-2.5-3B/7B. and Minitron-4B/8BB with temperature of 0.1, repetition penalty 1, and a maximum 1000 game steps. We conduct all experiments over three trials and report the mean score along with its standard deviation.
34

---

Published as a conference paper at ICLR 2026
Models
Pokémon Red
Models
Agent
Pokémon Red
Rank
Models
Input
Pokémnon Red
Rank
Owen 2.53BB
JJama-1.2-11
Llama-1.2 1.2-3B
0.0
0.0
0.0gee
8.5
8.5
8.5
Lluma
Rellection
Zero-shot
Planning
0.0sm
0.0cup
6.5
6.5
GPT-4a
Both
41.7gm
38.9
Qwen-2.5.7B
0.0
Ref-Plan
0.0gn0
6.5
Gemini
83.3-sa
Minimoo-488
0.0
8.5
Both
83.3
1.5
Minitron- 8B
0.0
8.5
Zero
33.3
3.5
GPT-4a
GPT-4o mini
38. Pare
0.0
8.5
GPT-4o
Reflection
Planning
2
Claude
Both
63.9
72.2o4s
3
63-mini
0.0
8.5
Ref-Plan Plan
38,9
Gemini-2.5 2.5-pro
83.3
Clande-3.7
63.
Table 25: Comparison across
Deegasek:R1
75.0
Table 24: Ablation study for agen- modalities on Pokémon Red. tic modules on Pokémon Red.
Table 23: Gameplay score on
Pokémon Red.
G.5 RESULT FOR POKÉMON RED
Table 23 presents the gameplay scores on Pokémon Red. Gemini-2.5-pm achieves the highest score of 83.3, followed by Deepseek-RR R1 (75.0) and Claude-3.7 (63.9). GPT-4o achieves a score of 38.9. Notably, all the open-source LLMS evaluated (Llama-3.2-11) BB33/B Qwen-2.5-3B/7B, and Minitron-4B/8B) recorded a score of 0.0, indicating significant challenges in playing Pokémon Red. Interestingly, o3-mini, which is expected to perform well due to its reasoning capabilities, also achieved score of 0.0. We observed that 03-mini exhibited a tendency to rely on its pre-existing knowledge or intuition rather than adapting to the game environment, such as consistently moving downwards based on a likely incorrect assumption about the exit's s location, leading to unproductive repeated actions. This highlights the difficulty some models face in grounding their reasoning within the specific context of the game.
Table 24 shows the impact of reflection and planning modules. For GPT-40, the ceffectiion-llanning agent achieved the highest score (38.9), followed by the 'reflection' agent (36.1), 1), and then the 'planning' and 'zero-shot' agents (both at 33.3). This suggests that reflection plays a crucial role in improving performance in Pokémon Red. In contrast, Llama-3,2 3.2-BB consistently scored 0.0 across all agent configurations.
The comparison across input modalities is presented in Table 25. For Gemini-2.5 pro and Claude-3.7, using both text and image imputs resulted in performance equal or better than using text input alone. This indicates that visual information can be beneficial in this environment. GPT-4o also showed a performance increase when both modalities were used (41.7) compared to text-only input (38 9), suggesting that incorporating image data aids the agent's decision-making.
H DARKEST DUNGEON
H.1 GAME DESCRIPTION FOR DARKEST DUNGEON
Environment. Darkest Dungeon is a turn-based roguelike role-playing game where the player manages roster of heroes as they explore procedurally generated dungeons filled with monsters, traps, and treasures (Red Hook Studios, 2016). Each hero has unique abilities and a stress level that influences their behavior during combat and exploration. Stress accumulates through continued exploration and battle, and heroes who reach high stress thresholds may develop afflictions that hinder or occasionally enhance their performance. The game emphasizes tactical positioning, turn-order strategy, and long-term roster management. Elements of randomness, such as attack accuracy, critical hits, and affliction outcomes, introduce uncertainty and require players to adapt their strategies dynamically. For implementation, we build our environment on top of rule-based bot (Kgleken, 2023), replacing its rule-based combat logic with the decisions made by L.LM agents. To access internal game states, we utilize the Darkest Dungeon Save Editor (Robojumper, 2023). Since the game does not support complete control via keyboard input, we employ the Xbox 360 controller emulator to inject actions from the LLM agent. We evaluate the agent's performance during the first embarkation mission after the tutorial, which we designate as our default evaluation setting. For consistency, and Crus ader, in that order, and equip the inventory with an additional 'Food' and 8 'Torches'. To ensure reproducibility, provide save file with this setup preconfigured. During the mission,
fix the party roster to include the P Plague Doctor, Vestal, Hi Highwayman
35

---

Published as a conference paper at ICLR 2026
System prompt
User prompt
You wr helpte al soleaue masegmsal with "harkee Thanrrof on the PC, Your is crmples the expedition while minimoinn the stre your altex much sc poscilic To achizve this decerine the bast nos action basel on the cancont task and lie gane sule.
eik Loe Earrated Artice hasl orger sonng still slan
(Mh Currert State
skill Targeting Reles:
CURRENTLY ACTING HERO:
tsing skill,
eneny's azastly the desianated ungetable enany runks munds
Name:
hiahwaystas
begating
Aleaye
tngarable
mrge
33,0350
Stass
(Eoongle le
only target cecmies rariss
select
namie ar2.1
More decals smnited this figure
Then IPP it the horo can will rake action. You may atrcck orrposs to change the enery formations.
Stile (Sieellwady
opened ven beger one recmy from manker (t. 2Jp (can ser from nanker
313 [AVAILABLE (BLE)
Level:
must zur cupput any deil sames action Actions must
Type:
rirger weing Sclawingg forms:
Accemus:
Damaggr-
lero
Effers. Bleed
beckwadd
canko. [2, from mikes
'wopp horo
[. [UNAVAILABBE from rack
itere, X. X. issages (I (l-aaed for slsts and ranlo You noat two such Inc
Lovd:
you beleve rultipl commandes one recded in segience Othenisee, se, just one time
Type ringel Acrency 45%,
Your firal outpst stat fellow eucctyy tha format:
Dareagr- -1%%
Crll. 7.5%
2Me Reasxing
Effers: HW
uoere porra ara cxplanation)
anal decrpptions
figare for sprcce
Resesald highhwmmmn))
HP,, 33.033300 Suuss,
No additiona
Casbrii (restali
lose doctor) Mank: 22/0-2.0
Streee 20.0 ALMBALLY
MOVED
Gome screenshot
FNEMY PORMATION
Hore Rabhiz (Dher vid 16, Resk-
HP: s.z, Stursod: Fabe, Alaady Movad, Fabe
Bleed: dduc. Biglt, 0 odus
Levds
Taze: Cantsckiledia..t
Staned:
Moved, Falss
Biglt, edar.
Rlase
Thrat Leve:
Tags CaeBokitedii Hi:
Figure 17: Action inference prompt for 'zero-shot' agent playing Darkest Dungeon.
dungeon exploration is handled by rule-based logic, while all combat decisions are delegated to the LLM agent.
We convert the internal game state of Darkest Dungeon into a structured textual description suitable for LLM input. The observation includes combat-relevant details such as the active hero's ' stats, available skills, party composition, and enemy formation. For each hero, we format key attributes (c.g., HP, stress, position, status effects) along with skill availability and target constraints, using symbolic descriptors extracted from a parsed skill configuration file. Enemy information similarly structured, including HP, rank, resistances, and threat indicators. The final text is composed of three parts: a detailed hero description with skill information, party summary with basic stats, and an enemy formation breakdown.
Observation-tor Text Conversion.
Action Space. At each decision point, the agent chooses one of four action types: 'Attack', 'Heal', or 'Swap'. For 'Attack' and 'Heal' actions, the agent specifies skill slot index and target index corresponding to specific enemy or ally. 'Swap' actions require the agent to provide the current hero's 's rank and swap distance. To allow for complex plans (e.g., swapping then healing), the agent may output up to two such structured command lines in sequence.
H.2 GAMEPLAY PROMPT FOR DARKEST DUNGEON
Figure 17 shows the action inference prompt used by the 'zero-shot' agent for playing Darkest Dungeon. The system prompt encodes task-specific knowledge, including (1) the primary objective of completing expeditions while minimizing party stress, (2) strict targeting constraints for combat skills
36

---

Published as a conference paper at ICLR 2026
Models
DarkestD
Rank
Models
Agent
DarkestD
Rank
Models
Inpat
DarkestD
Rank
Llama-3.2-1B
0.0ase
11.5
Zero-shot
47.5
Text
93.4215
2
Qwen-2.5-3B
Llama-3.2-3B
44.8am
47.5
10
9
Llama-3B
Reflection
Planning
47.31m
56.3+21s
6
GPT-4o
Image
92.2.mm
3.5
Qwen-2.5-78
88.827
6
Pef-Plan
57.01m14
Gemini
Text
93.7
1
Minitron-4B
0.0ce
11.3
Image
92.2=is
3.5
GPT-4o-min
Minitron-8B
63
81.3
8
7
GPT:4o
Rellection
Zero-shot
Planning
85. 1103
93.4215
82.0gs6
3
Claude
Image
Text
90.1-17
89.9
6
5
GPT-4o
13
2
Pea-Plan
91.6128
89.0
Gemini- 2.5 2.5
93.7
Table 28: Comparison across
Claude-3.7
Table 27: Ablation study for agen- modalities on Darkest Dungeon. tic modules on Darkest Dungeon.
Deepseck R1
91.7au
Table 26: Gameplay score on Darkest Dungeon.
based hero and enemy rank positions, and (3) the expected format for issuing valid commands. The user prompt provides the current game state, including the acting hero's 's stats and available skills, summary of the party, and the enemy formation. Some fields omitted for brevity, but the format mirrors the actual prompt used during inference.
H.3 EVALUATION METRIC FOR DARKEST DUNGEON
To evaluate the performance an LLM agent in Darkest Dungeon, we define a composite scoring metric that reflects progress, survivability, and stress management throughout the expedition. The score consists of three components: (1) the proportion of room combats successfully cleared, weighted at 40 points, (2) the fraction of heroes who survive the entire mission (out of four), weighted at 30 points, and (3) the remaining stress capacity of the team, also weighted at 30 points. However, the latter two components are only counted if the stage successfully cleared (i.e., if the first term reaches its full 40 points); otherwise, they are set to zero. To ensure fairness, the stress of any hero who dies before the end of the run is treated as the maximum stress (200) when computing the average team stress. The final score is computed as:
combats cleared f 40
+ 30 heroes survived ) + 30 (1- total stress
if stage is cleared otherwise
Score
combats
40
combats cleared
total combats
H.4 EXPERIMENTAL CONFIGUR ATION FOR DARKEST DUNGEON
For all open-source LLMs, we use temperature of 0.7 and repetition penalty of and set the maximum number of game steps to 200. We run all experiments with 3 trials and report the average score with the standard deviation.
H.5 RESULT FOR DARKEST DUNGEON
Table 26 reports the gameplay scores of various models Darkest Dungeon. We show that Gemini- 2.5-pro achieves the highest score of 93.7, closely followed by GPT-4o (93.4) and Deepseek-R1 (91.7), demonstrating strong capabilities across combat decisions and roster management. Among open-source models, Qwen-2.5-7B performs best, achieving a score of 88.8. In contrast, smaller models such as Llama-3.2-1BB and Minitron-4B fail make meaningful progress, often producing invalid outputs and scoring 0.0. A closer analysis reveals that Llama-3,2 1B frequently fails to follow the correct action format, while L.ama-3.2-3B and Dwen-2.5-3B tend issue invalid commands, such as using unavailable skills from incorrect hero positions or targeting unreachable enemies. These models also overuse 'Swap' actions, leading to inefficient combat sequences. Notably, many small models become stuck in loop when the 'Crusader' hero is affected by the 'Surprised!! status and repositioned to the back row. Since most the Crusader's skills are unusable from that position, the models repeatedly attempt invalid actions or swap ineffectively, wasting turns and failing to recover from the disrupted formation.
Table 27 and Table 28 present ablation studies agentic modules and input modalities in Darkest Dungeon. For agentic components, GPT-4o performs best in the zero-shot setting, with reflection and planning offering marginal even negative impact on its performance. Llama-3.2-30 3.B33B shows
37

---

Published as a conference paper at ICLR 2026
small but consistent gains when equipped with the planning module, though overall improvements remain limited. This suggests that large models like GPT-4o already possess sufficient planning capabilities for this task, while smaller models benefit only modestly from explicit agentic prompting. In terms of modality (Table 28), we find that providing image input in addition to text yields minimal improvement. For all models, performance remains similar only slightly better when image data is included, indicating that the agents primarily rely on the structured text input to make decisions.
I MINECRAFT
1.1 GAME DESCRIPTION FOR MINECRAFT
Environment. Minecraft (Mojang Studios, 2011) is an open-ended sandbox game where players explore world, gather resources, and survive by placing and breaking blocks. This environment based on the Mineflayer JavaScript API (contributors, 2013). Using Mineflayer, the agent can control a Minecraft bot through high-level JavaScript commands. We use Minecraft version 1.19 for compatibility, and to ensure consistent evaluation, we fix the world seed to 42 and initialize the bot at coordinates (604, 100, -823). The bot starts in survival mode with items and progressively crafts a target item using its in game observations and aavaerrpt-basee actions generated by the LLM. We select target items with varying levels of crafting difficulty: 'crafting table', 'stone pickaxe", 'furnace', "bucket', 'golden sword', 'diamond pickaxe', "enchanting table', and 'nether portal'.
Observation-to-Text Conversion. The Mineflayer API provides the bot with its state and contextual information from the surrounding environment. Specifically, the bot receives textual observations the form of: (Current biome, Day Time, Nearby blocks, Health status, Hunger status, Position, Equipped items, Inventory contents).
Action Space. The action space consists of JavaScript code that interfaces with the Mineflayer API. Following Voyager (Wang al., 2023), we expose the following set of control primitives to guide the LLM in generating valid and effective code actions for the bot using in in-context learning.
expl oreUntil (bot, di reccioon maxTi ime, ca 1 ba ck): Moves the agent in a fixed direction for up to Time seconds, or until custom stopping condition (defined callback) is satisfied.
minelkock (bot, name, count): Mines and collects up to count number of blocks with the specified name, within 32-block radius.
craftIten (bot, name, count):
Crafts the specified item using a nearby crafting table. placeItem (bot, name, position): Places a block of the specified type at the given position.
smeltItem (bot, itemName, fue count): Smelts the specified item using the provided fuel. Requires access to nearby furmace.
Ki 1MMbb (bot, mobName, t timeout): Hunts and eliminates the specified mob within the
time limit, and collects any resulting drops.
get ItemF temFFommhess: (bot, chestPosis iitossttin,, itemsToGet) Navigates the chest
the given location and retrieves the requested items.
deposi sttesttttmmmii Inteemnntooli Inteesttt (bot, chest Posestoosite Positionn itemsToDepositt): Navigates
to the given chest and deposits specified items into it.
I.2 GAMEPLAY PROMPT FOR MINECRAFT
Figure 18 shows the action inference prompt used by the 'zero-shot" agent for playing Minecraft. The system prompt contains gameplay-specific knowledge and guidance for action inference using Mineflayer APIs. It includes (1) the main task of the game, (2) control primitives, which is Javascript code template that should be referred to, (3) game observation in text format, and (4) the expected output response format with reasoning and code. The user prompt provides the current game state provided by Mineflayer APIs. Given this prompt, the LLM agent infers the appropriate Javascript code for action to complete the target task.
38

---

Published as a conference paper at ICLR 2026
System prompt
User prompt
lelyfal uosiann wakes Miscllayer janascript oode conpletle ay "Tak"!
proguara
(corail prinities)
Neathy blddet dis, goos. wlllgres, Jlaver,
Nerthy crtitee csorect ferteety Nowe
Al each mual converation, will siro
Heall: 25,/20
Cooe from the bus noundt....
Hengrr 20,030
Irsoaase cmee...
Prenun: 205925, yout it, pmacas
Blone...
Equpoeet [Noxs Novs, Nove, Noee, Vari, Nexe] irmmny (103) foar planted soidd deet 1]
Lime:
Neatly
Snbwady
Healit....
Teak: wooken
Poutter
Eqrioren....
Invensrr poiles
Cheea...
Tack
Coutest....
Chtigae...
foo tophin applicatles there
plas? doa the comples the task? What chat cecaaton ome imply?
Plan: Now to comglerr the tock sop by stgg You shoold atretion frvencory shoo rels what hove. The tadk compltenno clock sho besed on your fixal invertory. Code:
Write zn assre frnction raking the the only argareent
slove uedfll poogatt
posalee
divectly
direclly
unctitionbbee
speePor directyy
UUe skeelienebst, rume. paitien)
bockks Do betppacclllck direcly.
Use Lilbootlbe, sane, smoral) kill cado, Do
betattak dacatly.
Your fasstinn will he resed for sutiiin mon complex fanztione Therron, you sheeld mace it goent and sceablle You sloud not anko soong is xasgtion about fir investory ta ii be clarged at lits laec), and fhaooiiee you shoold always cleck wheche you lave the rogured haoe befoc wsing Doon If not you doull first progaen
Game screenshot
coliees the requiee and maee shove
from the roond section
excaned
lasel
Sarriiing defined
will
defise
call bet chell
inservetate
bot
Ty the cogine/aiibbo dirctions macDitaaee, callocky wiee canse fidd sunethitg. You slouad foquestly call thi befise minig bkodk er killnng cabss. You shoeld ielot direciiu randets time rnscad of communily using (1. B, I)
"maxiisnncee shoald alwiys
"bo, GatBlecks' and "wo BelBBock" Do not chost
sy nor witte iefinie kogs or recaniiv fractions.
ion butze botunse" regist:: evast latenen, You definiely not sood rans))
your
meniingtt way con
supeoal
Actiomied descited below.
Nel Actions
Explire....
Plat...
Code
POOOROOOOD
") jroscript oade
Figure 18: Action inference prompt for 'zero-shot' agent playing Minecraft.
1.3 EVALUATION METRIC FOR MINECRAFT
We evaluate agent performance using the success rate of crafting target item. Since Mineflayer APIs provide access to the agent's 's inventory, we compute the success score by checking whether the target item appears in the inventory at each game step. If the item successfully crafted and presented, the episode is marked as successful with score of 100; otherwise, O.
1.4 EXPERIMENTAL CONHIGURATION FOR MINECRAFT
For all LLMS, we use a temperature of and a repetition penalty of 1. The interaction with the game environment is limited to maximum of 100 steps. We repeat all experiments 3 times craft each target item, and report the average score and standard deviation.
I.5 RESULT FOR MINECRAFT
As shown in Table 29, 03-mini, Gemini-2.5-pro, and 3.7ude--.7-sonet performed the best on Minecraft, each obtaining a score of 75.0. All three models successfully crafted the following six items: "crafting table', 'stone pickaxe', 'furnace', 'bucket', 'golden sword', and 'diamond pickaxe'. However, they all failed to craft the two most difficult items: 'enchanting table' and 'nether portal'. In contrast, all six source LLMs failed to craft any of the target items, resulting in a score of 0.0. Notably, five of the models, except Minitron-8B. failed to generate any executable JavaScript code compatible with the Mineflayer API. While Minitron-8B was able to generate valid code sometimes to move the bot and mine wood, it failed to craft even the simplest item, the crafting table.
39

---

Published as a conference paper at ICLR 2026
Models
Mincoraft
Rank
Models
Agent
Minecraft
Rank
3.2-1-3.2-1R
0.0=0.0
9.5
7ero-sht
0.0:0.0
Llana momna-3.2. 3.2:3B
0.0-0.0
9.5
Reflection
0.0+a0.0
Llana 3B
Qwen 2.5.3B
0.0-000
9.5
Planning
0.0+0.0
Qwen-2.5-7B
0.0=0.0
9.5
Ref-Plan
0.0::0.0
Minitron-IB
0.0=0.0
9.5
Minitron-8B
0.0-0.0
9.5
Zero-sot
0.0:0.0
Reflection
50.0=0.0
1.5
GPT-4o
GPT-to-mini
46.0:17.0
Planning
13.0=0.0
GPT-4o
71.0+7.0
Ref-Plan
50.0=0.0
1.5
03-mini
75.0=00.0
Gemini-2.5
75.0++0.0
Claude 3.7
75.0:+0.0
Table 30: Ablation study for agentic modules on Minecraft.
Deepaeek-R1
41.7200
Table 29: Gameplay score Minecraft.
Sood
Buy Prics
Sell Price
Growth Days
Notes
Parsnip Seeds
35
Bcan Starter
Regrows every 3 days after first harvest
Cauliflower Seeds
80
175
12
Seeds
of extra yield
Table 31: Comparison available seeds Stardew Volley.
As shown in Table 30, the reflection module, which encourages the model generate improved code actions based on past failed attempts, significantly improved the performance of GPT-4o. However, the reffection-planning agent achieved a score of 50.0, which is lower than the default "skill- maiillmmnngemmntt agent score of 71.0 in Table 29. This suggests that the skill management module, which is responsible for storing previously successful code actions and retrieving them when needed, plays more substantial role in enhancing the performance of Minecraft.
J STARDEW VALLEY
J.1 GAME DESCRIPTION FOR STARDEW VALLEY
Environment. Stardew Valley (ConcernedApe, 2016) is life simulation and farming role-playing game. The player can engage a variety of daily activities such as farming, fishing, mining, foraging, and socializing with villagers.
Our objective is to evaluate an LLM agent's ability autonomously perform cemming-related tasks that maximize monetary gain within the first 13 in-game days (i.e., until the Egg Festival on Spring 13). Specifically, we focus on harvesting crops and strategically earning money by predicting high- profit crops and interacting with the in-game environment. The character begins Day 1 with 200 gold. Four types of seeds are available for purchase: parsnip seeds, bean starter, cauliflower seeds, and potato seeds. Each seed type differs in cost, selling price, days to harvest, and other characteristics. Table 31 summarizes the properties of each seeds. To make the task challenging and enforce planning under resource constraints, we manually set the player's maximum energy to 50 (default: 200). Using tools such as the hoe or watering can consumes 2 energy per use, limiting the number of tiles the agent can till or water in a single day. If the agent's energy drops 0 or below, they start the next day with only 26 energy (instead of 50). Furthermore, if energy falls -15 or below, the player loses 10% of their current gold and still begins the following day with only 26 energy. This constraint encourages the agent to prioritize actions and manage resources efficiently.
We run the game on the Steam platform and use the modding tool SMAPI (Pathoschild, 2025) to extract in- na-game states and implement custom actions. To send keyboard and mouse inputs, we use the pyautogui library on macOS and the AutoHotkey (AHK) library on Windows, following prior work (Tan et al., 2024). Since many in-game actions such as planting watering crops consist of multiple low-level actions (e.g., move up/down/right/left switch tool, use tool), we define a set of high-level actions to abstract these into semantically meaningful units. Each high-level action is mapped to a predefined sequence of board inputs and implemented via SMAPI scripts with custom keyboard bindings. We provide the save point used for our experiment to ensure reproducibility.
40

---

Published as a conference paper at ICLR 2026
Observation-to--ert Conversion. After each high-level action is executed, extract JSON- formatted game state via SMAPI, which includes information such as player location, current inventory, crop states in the field, remaining energy. and money. This state is then serialized into natural language description and passed to the LLM.
Action Space. We define compact action space consisting of 8 high-level actions essential for solving the task. These actions abstract away low-level controls and are defined as follows:
t iill soil (num tiles): Tills num tiles soil tiles to prepare them for planting. The tiles are selected in fixed order starting from the pre-defined position.
plant.seeds (): Plants all available seeds from the inventory into empty, tilled soil tiles. If the number of tilled tiles is less than the number seeds, only the available plots are used.
water_seeds (): Waters planted crops that have not yet been watered on the current day.
harvest crops () Harvests all crops that fully grown and ready to be collected.
sell_item (): Sells all harvested crops currently in the inventory.
buy item (item name, it count): Opens the shop interface, selects the specified item, and attempts to purchase the specified quantity. there is insufficient money, the agent buys as many units possible.
get.out-of (): Moves the character out of the house.
po._hose__ndd.s and_sleed_eepppppp eep (): Navigates the character back to the house, enters it, moves to the bed, and interacts with end the day.
J.2 GAMEPLAY PROMPT FOR STARDEW VALLEY
Figure 19 shows the action inference prompt used by the 'zero-shot' agent for playing Stardew Valley. The system prompt defines the agent's role an in-game assistant tasked with selecting the best next action based on the current situation and target task. It specifies strict behavioral rules, such as only using actions from a predefined set, avoiding repeated failed actions, and formatting outputs as Python code (up to two actions). The valid action set includes available actions with clear descriptions and constraints. The user prompt provides the goal and detailed context, including crop stats, energy rules, and the current game state (location, inventory, weather, etc.). It also includes the last executed actions and requires the agent to valid action output.
J.3 EVALUATION METRIC FOR STARDEW VALLEY
The objective in the Stardew Valley task is maximize the amount of money eamed during the first 13 in- in-game days, starting from Day 1. This period is a natural milestone the game, as the Egg Festival takes place Spring 13, where players can purchase high-reward crops such Strawberry seeds. We evaluate performance based on the net profit earned by the end of Spring 13, calculated as the difference between the final gold amount and the initial amount of 200 gold. We normalize the score using human expert baseline of 1156 200 = 956 gold, which we assign a normalized score of 100. Formally, the normalized score defined as
Score (X /staat (/ Fsaant (Iiinal Dstar ) 1/956,
where Pinal is the agent's 's final gold amount, Estart 200 is the starting gold, and Foracle 1156 is the human expert score.
J.4 EXPERIMENTAL CONFIGURATION FOR STARDEW VALLEY
For all LLMs, we use temperature of 0.0 and repetition penalty of 1. The interaction with the game environment is limited to a maximum 150 steps. For GPT-4o, we used GPT--4--0224-55. model for Stardew Valley. During LLM inference, the game is paused to prevent in tn-game time from progressing, which could otherwise alter the environment (e.g., a day transition). We perform each experiments three times and and present the average score with the standard deviation.
41

---

Published as a conference paper at ICLR 2026
Sy siem prampt
User prompt
the bitras acatur reugrace with Strkee valkoy on ts cuaped mandl vancus tode You
targt
You
pafft befor the meerrriig of 18s fiood ceeiiia cop ollerioo adtvetton but acd tye has difaces gaovt emo portinc cara and ing prics. Thpety food
twonacthaapee puspppepre ars buen die and balalt achir you suatuutiun cocndete in
dlg. lit hav ennor
tak,
sill Fur 171g fook you dast our Weg Vaudiloee foudd sic 12 tays cnd Big,
cathere X0% loace to yoil can
on the coreer of fhorack
daege bdoo baciin culsudd zaninng ras vit exeg enego cree b--1..
kis lerangy aud lilling walle weining sodl come men evigy por selion If
You chee singee arkkow flyfie onlo Remmatt aud willy mocr pineedeess to aneocale furiin
tili beflie lating ho bed cuil rereigh witlr coniiipeeaaiiii fyrnn dind you
selionn
cryp raly unllen be
by the soot ontions the serpet
seew wing lown Thong
you
preat
If por sweore horenin joua win ihe riuff benie, swi_stepp (ieppt)
week will enl duily
rody sill thom repert procss you ofter ootirn, sack on repaied Crey oettoatio
aut repora te pao sas wiliee agei sou biik prom votice facks
chere
vilil
if you Farnilkons, the tubk you MUST de laav bomus and go
cousad actian
den astiiastio sctie tyee farusa
oui of heast)) pire jook))
the Lionect
Desatake: Stari dessctur of bouse This ducciion wascaass the caso clararar of au earigiing though the The fnmeion by oeles cffect when be chesseer in made the
The kesled hore. The ellyer fign and 2010 ewery romaning
Burske bouu skved)
staning
lesust with pragett to shops This fanciis canomane Co act viving the cla slauc the bul and
reccniing luge slops
Copp groving
(Staci
lunser Weased Trail
tnu dase buy semonee anme sion, uiver)
Dacatatee: factharopes chepiaaattue
casedyy
wene, comdon fon liioo,
the aalda dlsiee, funatio
inllowing
Asx Paack
the has bo brught (cOOOCts: Thrmay Souds "Bea Statort, "Caahtlower Seal?? foudf))
sures
Woasingg (Rade:)
-ikm, pase The marter same to bo beaghs
Phdlacc (Stukc
Decat sioe sall dicess iv ievenvcy This duccic sumwar fs oaeas the dayy cantiioned ualk all
You dowiltorlyy
sepodd fhr Eornit dow. had belia, and cupeticnnnent
tavenstyy
H4d Twreeny
Phactioe valkom tloo
feniioon unnooratiiiat thogivee madeer aft kil bosted phopdddi pestiies Thi Fet by work wauc the diancir is the dasa acca
raen Jll his wlow wal ao
For riee pland wowha)
Docatste:: Exscion plants anstabll suds dres tanentry adi opensic under
Game scrvanshot
sunds only avadable plues will be and The clanater mar be la the farn uraa de da dractica week reveniayy Faneiteeen
Fhe ricee sikke joede)
reectes work Neustppcee: welend, this fanciims witieennt ing.
De Devrtgion: Humort expo whick reddy harvert Thes finetio work when chenster the
Figure 19: Action inference prompt for 'zero-shot' agent playing Stardew Valley.
Models
Stardew
Rank
Models
Agent
Stardew
Rank
Models
Input
Stardew
Rank
Llama-3.2-1B
0.0eas
10.5
Zero-shot
0,0+00
6.5
Text
81.4s4
1
Llana 3B
Qwen
0.01as
0.0gaz
10.5
10.5
Llama-3B
Reflection
Planning
0.0gae
0.0
6.5
6.5
GPT-4o
Image
Both
41.9=221
0.0-us
8,5
6
Qwen-2.5-7B
0.0gas
10.5
Ref-Plan
0.02a0
6.5
Minitron-4B
0.0eas
10.5
Text
59.2-mm
3
Minitron-BB
0.0gas
10.5
Zero-shot
40.5
Gemini
Image
7.6=99
7
GPT-4o-mini
16.1+m7
7
GPT-4o
Reflection
Planning
64.6
18.3253
2
Both
60.0ese
2
GP
81.4_4x
Ref-Plan
95.7=5t
Text 53.62m0
4
GPT.5
92.3_46
Claude
Image
0.0=ao
8.5
03-mini
Both
49.8+10
5
Gemini-2.5
59.2+mu
4
Table 33: Ablation study for agen-
3laude-3.7
53.6120
6
Deepseek-R1
66.1eua
tic modules on Stardew Valley.
Table 34: Comparison across
modalities on Stardew Valley.
Table 32: Gameplay score on Stardew Valley.
J.5 RESULT FOR STARDEW VALLEY
Table 32 presents a comparison of LLM performance in Stardew Valley. Among the models, GPT-4o presents the best performance, followed by Gemini-2.5-pro Open-sourced LLMs fails to earn money, primarily for two reasons; (1) failure perform valid actions (all models excepts Qwen-2.5-7BB
42

---

Published as a conference paper at ICLR 2026
(2) poor crop scheduling, resulting in crops not being ready for harvest Day 13 (Qwen-2.5-7B) Under the imposed energy constraints, the optimal strategy is to plant Parsnip Seeds every four days, as they yield the highest profit. The player should purchase and plant many seeds possible on Days and 5, and exactly 24 seeds on Day 9. Planting more than 24 seeds on Day 9 depletes the player's energy during watering, leading insufficient energy the next day and ultimately failure to harvest on Day 13. None of the LLMs, including the API-based models, fully followed this optimal strategy. In particular, most models frequently selected suboptimal crops such as potato seeds, which contributed to their lower performance.
Table 33 shows the effect of different agentic modules on performance. In the case GPT-4o, the planning module has a significant effect, as the task requires accurate seed selection and scheduling. In contrast, Llama-3.2-3B model fails to make profit across four agent configuration, indicating that the underlying model's capabilities are a limiting factor regardless of agent design.
Table 34 summarizes game performance under different input modalities. Notably, all three propri- etary LLMs fails to achieve strong performance when only vision input is available. This underper- formance is mainly due to the difficulty of extracting structured information from a single screenshot (see figure 19 for an example). Although the screenshot contains rich contextual information, it also contains a lot redundant content, and critical information occupies only a small portion of the image. For instance, watered plants appears slightly darker than dry soil, making it difficult to distinguish visually. Similarly, the current day, a crucial cue for planning, is indicated in a small font in the upper-right corner of screenshot. When both text and image inputs given, GPT-4o and Claude-3.7-Sonnc exhibit performance drop, while Gemini-2.5-pa shows improved performance. This suggests that only sufficiently capable models can effectively integrate multimodal information, while others may struggle with modality fusion or become distracted by noisy visual inputs.
K STARCRAFT II
K.1 GAME DESCRIPTION FOR STARCRAFT II
Environment. StarCraft Il is a real-time strategy game where players gather resources, construct buildings, train units, and command armies to defeat opponents. The environment features partially observable map, requiring the agent to explore and gather information about the opponent's 's actions. For implementation, we adopt the BurnySc2 / fpythonn scc environment (BurnySc2, 2017), a Python interface widely used in the reinforcement learning (RL) community. Note that the library supports the raw scripted interface without a graphics- based interface. The environment supports various official maps and game modes. For default evaluation (as in Table 3), use the 'Ancient Cistern LE' map with the agent playing as Protoss against the built-in Al bot (Zerg, Hard difficulty; employing timing build order strategy). The environment allows testing across different races, maps, and difficulty settings. For instance, the "Babylon LE' is used to assess intra- intra-game game generalization in Table
Mnservattonnto-Tex Conversion. The BurnySc2/ hotthottoooooooeo sc2 environment provides the agent with observations capturing the current game state and context. Specifically, the observations include: [Resources, Buildings, Units, Research Progress, In In-progress Actions, Enemy Information, Game Time}. We convert these observations into a concise text summary; an example summary is shown in Figure 20.
Action Space.
Following the BurnySc2 2ttnyss22/ptthonsc222 implementation, We define the action space as a discrete set of 72 high-level commands specifically for the Protoss race. These include unit training (e.g., Probes, Zealots, and Stalkers), building construction (e.g., Pylons and Gateways), research upgrades, scouting, multi-unit attacks retreats, and special abilities (e.g., Chrono Boost). complete list of these commands is provided Figure 20. At each game step, the agent generates a list of five actions, which are executed in order.
K.2 GAMEPLAY PROMPT FOR STARCRAFT II
Figure 20 shows the action inference prompt used by the 'zero-shot' agent for playing StarCraft II. The system prompt contains gameplay sppecffccc knowledge and detailed instructions for action inference tailored to the Protoss race. It includes (1) the main task and game context, (2) a comprehensive
43

---

Published as a conference paper at ICLR 2026
prompt
Use prompt
You aas wipftt al au miined play Suatio
ewr Canert cine
Cammerty you are playing scttatess thery's mos is Zeng
Summary Acesss gams time, cur currer cteoton follower
You will be given a sizko san seay in gane.
lased on the givce anoe we wese you to anslose the pan- propenicee provode qpartic straleg: aggesiion, and suggest the oas sukibli actions for comert stustion.
Reccurcos:
Gare time or 35, Worker capply: 20, Mineral 75, Supply left: Supely ca: Supply med: Ary eapby:
beief
Builligge
langs,
teprogeus.
Pylon
halldings Worp
Ilfor petentia
indertte enemy's
haned arailabic informaion.
5. Propore emenn curoent
eneny's and capttalis sbength..
Unite
coute 20, Zesker connt
Actiom:
Boad oo the given information we wone you to make 5 actorable and epeciti decioone to fallow coment anargy The ontinn decisions ahoald be estuated foon dor ACTION DICTIONARY Ieln..
In Progres
Unit ponducing:
Guddllies
ing pols coust
exren rxssuree staus after excrsting peoriou sctien.
sepend
Asalysi
5. 1-4 mimee Remember them artive decisone will excettd chrosiegiaaityy
ACTION DICTXINARY
(TRAINN PROBE D. TRAIN ZEAL CTTY L. TRAIN ADDPT: 3. TTRIIN STAL KER" 3. TRAII SINTRY
eld Rasceling ig
TRAIN HIGHTEMPLAR:: 3. TRAIN DARKTEMPLARE AARLARRAA 6, TRAIN VOLDRAY 3, "TRIIN CARRIER: &
I: [Cuneni (ACTTO]][ACTION] [Cool] (Availabiiay Uoduaed Besneroe
ARN TEMPREST
ACTE" IS ITRAIN PIDINII 1I, TRAIN METRIRSSHTP
OBSERVER: 13. "TR IMMORRAAL 14, TRAIN WARPIRISMT TRAIN COLOSSUS: TTRINN
DISHLIPPON- "MORPII BON- IN 'ILLLD LON': WIILD ASSINC SOSIIILATOR-AA 'BUILD NEXUS
BBULD
CYBERN
'BUILD
TWILIGHTOOUNCTE
FLEETBE
SHLELDBATTERY: ANNNNNNN WE ARONSLEEEELLLL RESEARCH
SACTIIN- SACTION
WARRGATESS
PROTURS RWIA PONSI EVELLT 2% 'RUSUARCCH PRO TOSSAIRS AF ONOSS BV ILL" RISEARCH
PROTOSSALE SASRARA
BESLEELEL 3E, RESEARCH PROTOSS AIR ARMORR SLEVEL 29, RESEARCH
PROTEBSAL KAR KOISLLIN EE 40, TEE ALRRII ALE ALETTIPEE PREECINGTT ALK' 41, MISHARLH
Game screenhoot
BL INKTEEET 4T RESE ARCH ROE 43, 'REST ARCH PRO PROTOSSGROUNDWAA.R PAPONSIEVEI IY 41,
RESEARCH ROT BOTOSSGGOUNN DWEAFONSL EVELT: 45, RESEARCH
PROTOSSSUR INPRTAAPONS TVEE M. RETI SROTOOSSRRINNAAAIIAS
DESARRCH ROTUSSC BOUN MOR
ARCH FROTUSSGROUND RMORSI INTEE
RESEARCH POTOSSSHIE
EESHARCH PROTOSSHHELDSLLE
RESEARCH
3 S2,
EXTENDEDTHENNE IR
PRAVITICDIIEE
RESEARCH TEMPESTGR CONNNTAAAIIPOOO
SCOUTING PROME SCOUTING
OBSERVER: LL SCOUTING ZEALOT SCOUTH ING PHOENIX MULTEATTACK 64, MULT-
RITREATT CHRRODRROOST NEXUS 96 WURONDOOOS CYBURNETI USCDRE
CHRONOBOOST TWIL CHHTCOUNCILL &R, &. CHRONOBOOST STARGATE, OR CHRONOBOOST FORGE! IN, TIMPLY AUTRRNN: 73)
Figure 20: Action inference prompt for 'zero-shot' agent playing StarCraft II.
action dictionary listing all possible unit production, building construction, and research actions, (3) the current game status summary including resources, buildings, units, and ongoing actions, and (4) the expected output response format that guides the agent to provide step-by-step analysis, reasoning, and 5 concrete actionable commands with cost and resource availability considerations.
The user prompt provides the current game state with detailed information on resources, supply, buildings, units, and ongoing unit production. Given this prompt, the agent infers the appropriate Protoss-specffic actions to optimize the gameplay strategy against a Zerg opponent, focusing on resource management, army composition, and tech progression to counter the enemy effectively.
K.3 EVALUATION METRIC FOR STARCRAFT II
Single-Agent Play. In the single-player mode, the agent competes in a series of matches against the Al bot opponent. It plays up to 4 matches, continuing until it either wins or loses. The evaluation metric is the win rate, calculated as:
Number of Wins
Score
100
Total Matches Played
Multi-Agent Play. In the multi-player mode, we do not use win rate as the performance metric, since only a single run is conducted per match. Instead, we measure the performance by the army supply difference between agents at the end of the match. The difference is calculated as the sum of each unit's count multiplied by its consumed resource cost, reflecting the effective army strength. The winning agent receives a positive score equal to this army supply difference, while the losing agent is assigned the negative of this value. This scoring method captures not only victory but also the margin of the win.
44

---

Published as a conference paper at ICLR 2026
Using these difference- based scores, we then compute Elo ratings for all agents following the Bradley- Terry model (Bradley & Terry, 1952), following the approach used in StreetFighter III multi-agent evaluation in Section C.3.
The resulting army supply difference matrix and Elo scores are presented in Figure 4b.
K.4 EXPERIMENTAL CONFIGURATION FOR STARCRAFT II
For all LLMs, we use a temperature of 0.1 and repetition penalty 1.0. During LLM inference, we pause the StarCraft II game environment. Interactions with the game environment are limited to maximum of 1,000 steps. For single-agent play, we repeat the experiments times and report the average score along with the standard deviation. For multi-agent play, we run single experiment and report the Elo score.
K.5 RESULT FOR STARCRAFT II
Single-Agent Play. We evaluate agent performances in single-agent environment, all operating in a ref-plan setting. Table 35 shows a significant performance gap between open-source LLMs and proprietary LLMs. Among them, Llama-3.2-IBB3BB models mostly repeat simple actions like scouting and mining minerals, without showing much strategic planning. On the other hand, proprietary LLMs, except for 03-mini, achieve over 50% win rate. Notably, GPT-4o and Gemini-2.5 2.55ppo- won all four matches against the AI bot. They demonstrate strategic behavior by appropriately allocating resources over time line with the game's 's progression.
Table 36 presents ablation studies on agentic modules. As previously mentioned, Llama-3.2-30 models fail to manage matches effectively regardless of ablation settings, resulting a 0% win rate. In contrast, GPT-44o demonstrates remarkable planning abilities, which are critical in StarCraft II given its real-time strategy nature. other words, long-term planning to sustain strategies over time plays pivotal role in securing victories. Interestingly, models relying solely on reflection perform worse than zero-shot, suggesting that reflection without proper planning may actually degrade performance.
Table 37 presents the results of modality experiments. Surprisingly. for GPT-4o, using both text and image inputs results in decreased performance, and Gemini and Claude also show no improvement. This implies that agents and the addition of image input may introduce challenges multimodal reasoning.
make sufficiently accurate decisions based textual observations alone,
Multi-Agent Play. We evaluate agent performances in multi-agen setting by conducting pairwise matches among seven LLMs (excluding Gemini-2.5-pro. all operating under a ref-plan configuration. The results are summarized in Figure 4(b). To ensure fair comparison, both competing agents consistently use the Protoss race in every match.
Interestingly, unlike the single-agent evaluation where Claude achieved only a 50% win rate (Table 35), Claude outperforms GPT-4o and attains the highest Elo rating in the multi-agent arena. Even more surprising is that Minitron-8b, which had 0% win rate in single-agent play, defeated GPT-4o, 03-mini, and GPT-4o-mini, earning the second highest Elo rating. This discrepancy suggests that the presence of multiple intelligent agents can significantly alter game dynamics, potentially due increased strategic diversity or emergent adversarial behaviors. We also acknowledge that a single evaluation episode may have introduced some bias the observed rankings.
L SLAY THE SPIRE
L.1 GAME DESCRIPTION FOR SLAY THE SPIRE
Environment. Slay the Spire is deck-building rogue-like game where the player ascends procedu- rally generated three-act tower. Each act consists of a branching map with various room types such combat encounters, shops, treasure rooms, rest sites, and random events, ending in boss fight.
Our goal is to evaluate an LLM agent's ability to reason over strategic choices in a stochastic, multi-step environment. Specifically, task the agent with playing as the Tronclad character under standard rules (no ascension levels) and aim to defeat the final boss at floor 50. The LLM agent is responsible for key decision types: (1) choosing which cards play during combat, and (2)
45

---

Published as a conference paper at ICLR 2026
Models
StarCraft
Rank
Models
Agent
StuCCraft
Rank
Models
Input
StarCraft II
Rank
Qwen-2.5-3B
Llama-3.2-18
L.2-3-3.2-3B
0.0aza
0.0aza
9.5
9.5
Liama 3B
Rellection
Planning
0.0gn0
0.0son
0.0gn0
GPT-4o
Both
Text
100.0.ae
50.0gm
5
2
Qwen-2.5.7B
0,0aca
9.5
Ref-Plan
0.0amo
Gemini
Text
100,0s00
2
Minitroa-IB
0.0aza
9.5
Both
100.0.40
2
Minitron-SB SB
0.0aca
9.5
Zero-shot
50.0asta
GPT-4o-mini
75
GPT-4o
Reflection
Planning
50.0asta
0.0so0
Claude
Both
Text
50.0
50.0
5
03-mini
GPT-4o
100.0
1.5
Ref-Plan
100.0
Gemini-2.5 2.5ppi--.55ppo
100
1.5
Table 37: Comparison across
Clande-37 3.7
50.0g50
4,5
Deepsoek- R1
Table 36: Ablation study for agen- modalities on StarCraft II.
tic modules on StarCraft II.
Table 35: Gameplay score on StarCraft
selecting rewards after battles. All other game decisions, such as map navigation, non-combat events, and potion usages, are handled by simple rule-based policy.
The game runs on the Steam platform, and we use modding tools to enable communication between the game and external agent. In particular, use BaseMod (Bug Kioocht, 201 and ModTheS- pire (Bug Kiooeht, 2018b), along with a modified version Communication (Forgotten Arbiter, 2019), which enables state extraction and action input via standard input/output streams. Our modified version also extracts detailed in-game information, including full card and relic descriptions.
To ensure consistency and reproducibility, we fix the game seed for all This guarantees the same map layout, encounters, and card offerings. Additionally, since the original game unlocks card pools progressively as the player completes runs, we pre-unlock all cards using the Unlock Everything mod to make the full pool available from the start.
Observation-to-Tees Conversion. After each action, extract a JSON representation of the current game state, including player status, opponent status, current cards in hand, and relics. These states are serialized into a concise natural language summary that passed to the LLM.
Action Space. Our environment defines two categories of actions: combat and card selection. During combat, the agent can either play a card (PLAY) or end its turn (END). the card selection stage, the agent can choose a card reward (CHOOSE) or skip the reward (SKIP).
PLAY CARD_INDER Play non-target card from the hand at position CARD_INDEX.
PLAY CARD INDEX TARGET INDEX: Play targeted card from the hand position CARD INDEX, targeting opponent at TARGET INDEX.
END: End the current turn.
CHOOSE
Select the card reward at position CARD.INDEX
SKTP: Skip the card reward.
L.2
GAMEPLAY PROMPT FOR SLAY THE SPIRE
Figure 21 presents the action inference prompt utilized by the 'zero-shot" agent to play Slay the Spire. The system prompt describes the agent's role strategic player in the game and outlines key game mechanics, including block, energy, card draw, and enemy intents. Detailed game states, such as player status, relic, card, and opponents are provided the user prompt.
L.3 EVALUATION METRIC FOR SLAY THE SPIRE
The primary objective is to reach and defeat the final boss located on floor 50. Accordingly, the baseline score determined by the highest floor cleared. For example, if the character dies on floor 43, the score would be 42. Since each floor has varying difficulty levels, and bosses-app bosses-appearing at the end of each act act-pose pose significant challenges, we assign bonus points to boss defeats. There are three major bosses and we grant additional 50/3 points for each boss defeated. Formally, the total score is defined as:
50
Score = (# of Cleared Floors) +
x (# of Bosses Defeated)
(1)
3
46

---

Published as a conference paper at ICLR 2026
System prompp
Lser pronaee
er comica playe the game Say spec:
cou actua tuulie
celarcciaaaaattee CHOOsE
Provide decraveetn eenng for shout the sute ver cump Be Tick shoud sugu Dc
gaic
COMBAT STATE tham
Beasuinn kitkick by siepaal cow des wng dene whe
Clow IRONC AD
Gane Mivkk fioe, 1iP
Maxi
Ao out of much nans anlaa aucittc by offict, pager soaciie aveuo sad drava caste-
Eastor
Nomaee will comtrenggy
stack
whice plyye offer soales
eecure everyy
Bele
When CARD waln EETOO AIB sor by chonee andd
Save Bennng Mive
Deuttake: combar
Thee M1157 clucss acdise onl foes dhe phua yu auhie act Aup action unaiia dia unfictyy fediddos ple
Cucail hadd
berigg resonce
SMaaa Upperas
love enliippecei otten, teee sangi revinn thame aemuton, 12.1MMM
"Tppe stick
Apply Weae.
Valacaalle
Tan Tupest
Nome Sirii
Game screenshot
Saaae Stulia
"yppe usact
Toreet
Carl Lnder
Type
Torget
Nome Worm
balf Fales
Chroca Radd
Stove age
Yald sedionc
AV saad Talee Jalec- END
by sage al dndtleelllee aepertceenetttt cet Boowwing
Figure 21: Action inference prompt for 'zero-shot' agent playing Slay the Spire.
L.4 EXPERIMENTAL CONFIGURATION FOR SLAY THE SPIRE
For all LLMs, we use a temperature 0.0 and a repetition penalty 1. Interaction with the game environment is limited to maximum of 200 steps. The game is not paused during LLM inference, as the game state does not change over time. All experiments carried out in three runs.
L.5 RESULT FOR SLAY THE SPIRE
Table 38 shows comparison in gameplay performance between open-source LLMs and proprietary LLMS Slay the Spire. Most open-sourced models fail to make meaningful progress in the game, resulting in near-zero scores. In contrast, API-based LLMs demonstrate significantly stronger performance: Gemini-2, 1-mini-2.55pra achieves the highest score, defcating the second boss two out of three runs. The inherent stochasticity of the game contributes to high variance across all proprietary models.
Table 39 presents ablation study evaluating the impact of agentic modules, reflection and planning, on performance in Slay the Spire. For the weaker model (Llama-3.2-3B) none of the agent variants achieve any meaningful progress, indicating that architectural changes alone insufficient without
47

---

Published as a conference paper at ICLR 2026
Models
SpaySpire
Rank
Models
Agent
SlaySpire
Rank
Models
Input
SlaySpire
Rank
Llama-3,2-1B
0.0ess
10
Zero-shot
0.0zas
Text
23.6-12
3.5
L.ama-322-3B
Qwen-2.5-3B
0.0ess
0.0gre
10
10
Llama-3B
Reflection
Plunning
0.0gne
0.01as
6,5
6.5
GPT-4o
Both
23.61ma
3.5
Qwen-2.5-7R
5.0gse
6
Plan
0.01as
Gemini
Text
$1,9asi
1
Minitron-4 4B
0.0gst
10
Both
26.2+104
2
Minitron-8R
0.0gse
10
Zero-shot
12.3
GPT-4o-mini
7
GPT-4o
Reflection
Planning
23.2
26.2+194
Claude
Both
Text
15.0+ao
9.7
6
5
GPT-4o
23.6az21
3
Ref-Plan
23.6+z21
03-mini
15.0
4.5
Gomini-2,5 2-ppro
$1.92n9
1
Table 40: Comparison across
Clando-3.7
15.0
4.5
Doepsock-RI
24.9am
2
Table 39: Ablation study for agen- modalities on Slay the Spire.
tic modules on Slay the Spire.
Table 38: Gameplay score on Slay the Spire.
strong base capabilities. In contrast, GPT-4o benefits substantially from added reasoning modules: the reflection variant achieves the highest score, while both planning and reflection-plannio agents also outperform the zero-shot baseline. Notably, the planning module relatively less effective, which may stem from the fact that optimal actions highly sensitive to the opponent's intent, information that is only partially observable and difficult to predict multiple turns ahead. These results highlight that while base model capacity is prerequisite, structured reasoning routines further enhance gameplay performance in complex decision making environments.
Table 40 compares model performance in Slay the Spire when using either text-only input combination of text and vision inputs. In all three proprictary models, adding visual input does not improve performance-ani in fact, often leads degradation. This outcome is not entirely surprising, as crucial gameplay information, such as detailed card effects, relic descriptions, and power mechanies, is often absent in the game screenshot. As a result, the image input fails to provide meaningful utility and instead introduces ambiguity or redundancy, effectively acting as noise rather than useful context. These findings suggest that for structured, information-dent environments like Slay the Spire, high-quality textual representations remain the most reliable modality for LLM agents.
M
BABA IS YOU
M. GAME DESCRIPTION FOR BABA IS YOU
Environment. Baba Is You is a puzzle game in which players must discover and understand ev- ery rule and mechanic on their own, apart from the basic movement keys ('left', 'right', 'up', and 'down') (Hempuli, 2019). The game's s defining fea- ture is that the text tiles forming the rules can be pushed around, allowing the player to rewrite those rules the fly. Every valid rule sentence must contain a verb (e.g., 'Is' and 'Has'), and text tiles with colored background (e.g. 'You', 'Push' and 'Win') ) cannot serve as subjects. A level is cleared when the object designated by 'You" touches the object designated by 'Win'. For instance, with the rules 'Baba Is You' and 'Flag Win', the player wins as soon as Baba touches the flag. For imple- mentation, we instrument the Steam edition with a lightweight Lua mod that, after every player move, dumps the full internal game state, i.e., the coordinates of every object, to JSON file. The Lua modding script uses mod hook functions provided by the game developer, which are triggered at specific points in the game's 's code. The agent outputs delivered to the game via simulated key presses using the pyautogui library. We use Level 1 Where do go?, shown in Figure 22, as our default evaluation setting (for Table 3).
PAUSE
Eat
KC
Figure 22: Level 1 of Baba Is You.
Observation-to-Tess Text Conversion. We present the LLM with a textual description of the game state in parts. First, we list the current (I. y) coordinates of every object in the level. Second,
48

---

Published as a conference paper at ICLR 2026
System prompt
User prompt
Vee we on Al bedeove playing the purnfe game *hibhhs lo You on PC, Your objezive esecatz requeree of actions to comples the movementt from the commne eittodk
Comer caur
Level whore do go?
Disretsions 24x18 (axx x. 23, unx y:
Bula Is You Gone Mocherics
Ulimat poal is make abrect you curzal (VOU) loach amothing defined as WIN.
Oftecte
Rules are fomedd ONLY deez test blodos imaagedd connecstiv di (diaedly adjacont with so sapa)
-objettgracc posisams (xx yoll, oc3, yo y: 1o
(st muterry [PROISaTY)
-ohjurrost prpettoon[[er [[rk, yo),, y:1),
amonged honoorrally
right wamrally (up
-objarttie
poolion
Connras propentess YOU (comollatie) PUSH (oen movell STOP (blecks covvner)) WIN
testis
IDx (x77,):44x y.. 1211_44).
conditio))
testtyuu pastione (xx()
Test blaces themacives car be prabed
teaacllag posttics Exck y:1)
-Ohes crartig new nite or breaking coditing rale is key solving parntes
objorr flag postioos [a:17, y-9)
The oolation may regirm mutropie nus changes
maatosd at poeitons [[x-12 yr131]
It instanes of in diject with YOL propert are distittted you oue so lonaer aove
leat sopt el pipostions [xx.12, y:15
Coonsnos solution pettauss inclade: couvelling one otjest to mootee. changing whas objectis YOU, ookking something WIN, negating ralos creating dumooos.
test was al positions (0:15,y.77
objscnhhter at positions [so yr12))
Proking
Novigne gorion (MPORTANTY
poah
your
objar (YOU)
adjocett mite disection
WALL STOP
YOU (xDC y-Y-1)
object
preting DOWNS YOU an arocc 1-41 prstox obrect (1XX.gr1)
Last exeeated action:
boor prshing LIB 1: YOU peddin y:1)) pankic cbject
nigh:
prshing RIGHT YOU DCX X.1. y) otsees at (XX. y.Y)
2.
IMPORTANT When girting into posttion pash, avoid mosing divectly towank produabll objects obyects
leh
Eaazmples if yourre at bcR 1il
wast so pash eot (x).,)
WRONG Movina, left fins workd accidonnally push ohe-ohjac lefl
Pleaae reqpaad with dae folloving fentzi:
-OOEECTT: Mese door find to D-8 yrl.3, then left to (c7, y:12 thon MMdiiplc abyerts in line can be produed ermalnnoosaly if none STOP
Resoning
(Yeur step- hy-p reacoring hand)
Coondratz System
action
disccion
inzeomux dwwn, damrossscp
Think stop by doot the bex iequesee of actioss complete iuvveneent (lore the sabtask).
1. What object an comcetly coundling Chas the YOU propert floms octive rales) and where an 1?
2. Where is the obrect need to move (iem the wabtak??
3. Where do need to to meve dhe object in wanted divection?
Game screenshel
the compless parh mach this posltcr.
Whet seguence aettoe would you
most efficeenay?
Aue there are obitaclss along the polk?
IMPORTANT: moving towadd puslable objeet cause as usistented pask??
PP PRESE
Clomider
your
happen collisions néee diongre?
Ane there
Would socidenaslly prsh wmnthing?
Egg maving 3 foon DcR, yill) seaas musing throuh (v.5, y. 10), DCR, y.9), y.8), Siace there is
wall whide STOP (nc,, 99, Bhis adtion invelid.
Actian Permat: You epecily a sequence of actions where cuch actios incluke directions and optional cumberr of
sags.
Rommat Virectioe (supel" dedusiouo
Avaliible dinections up, down, lell, right
EA
Exarpies.
Simple
soquenee:
Figure 23: Action inference prompt for 'zero-shot' agent playing Baba Is You.
manually parse the state to extract all active rules and append those rules to the prompt. The combined description forms the model's observation.
Action Space. In the 2D grid environment, the agent can move 'left', 'right', 'up', or 'down'. At each step, the agent outputs a finite sequence of these directions, which translate into consecutive key presses and send to the game.
M.2 GAMEPLAY PROMPT FOR BABA Is YOU
Figure 23 shows the action inference prompt used by the 'zero-shot' agent for playing Baba You. The system prompt encodes key game-specific knowledge, including (1) the primary objective of touching a 'Win' object with a controllable (" ('You') object, (2) an explanation of how rules are formed and manipulated via text block arrangements, (3) crucial pushing mechanies with coordinate- baasdd guidance to avoid unintended interactions, and (4) the expected input-output format for the LLM. The user prompt provides the current puzzle state, including the map dimensions, object locations, and active rules. Given this prompt, the LLM agent outputs sequence of directional actions.
M.3 EVALUATION METRIC FOR BABA IS YOU
To evaluate the performance of an LLM agent, we define a hierarchical scoring metric that rewards meaningful progress toward solving the puzzle. As shown in Figure 22, a key prerequisite is breaking the rule 'Wall Is Stop'. which enables movement out of the closed area. The second subgoal is creating winning condition, such as forming the rule 'Flag Is Win', but this is only possible once the
49

---

Published as a conference paper at ICLR 2026
Models
BobalsYou
Rank
Models
Agent
BahalsYou
Rank
Models
Input
Babals You
Rank
Jama-3.2-18
6.7aus
12
20.0.co
Text
20.0=se
Qwen 2.5-3B
Llama-3.2-3
13.3gms
20.0gm
10.5
6.5
Llama-3B
Reflection
Plamning
20.0
20.0
4.5
GPT-4o
Image
Both
6.7am7
20.0
6
Owen-2.5-7B
Ref-Plan
20.0=co
4.5
Minitmo-415
20.0gge
6.5
73.3
Minitron SB
20.0go0
6.5
Zero-shot
20.0=co
4.5
Gemini
Image
20.0.sa
GPT-4o-mini
IL5
10.5
GPT-4o
Reflection
Plamning
20.0
20.0
4.5
4.5
Both
86.7
GPT-4o
63-mini
73.3
20.0
6.5
1.5
Ref-Plan
20.0
Claude
Image
46.7.1.3
20.0
Gemini-2.5-pon
73.3 442
1.5
Both
20.0.ae
Claudo-3,7
46.7.a42
3
Deepseck-RI
20.0gmm
6.5
Table 42: Ablation study for agen-
tic modules on Baba ly You.
Table 43: Comparison across
Table 41: Gameplay score on Baba Is You.
modalities Baba Is You.
wall constraint is removed. Each subgoal provides 20 points. If the agent clears the level by having the 'You' object touch 'Win' object, it receives full score of 100, overriding subgoal rewards. The final score computed as:
100,
if level cleared
Score =
40,
if "Wall Is Stop' is broken and 'Win' rule is created
20,
if 'Wall Is Stop" is broken only
M.4 EXPERIMENTAL CONFIGURATION FOR BABA IS YOU
For all 6 open-source LLMs, we use temperature of 0.7 and repetition penalty of 1, and set the maximum number of game steps to 30. We run all experiments with trials and report the average score with the standard deviation.
M.5 RESULT FOR BABA IS YOU
The performance of different models on Level of Baba Is You using the 'reflection-planning agent shown Table 41. We observe that o3-mini and Gemini-2.5-pro achieve the highest score of 73.3, significantly outperforming all other models. Apart from these two reasoning models, only Claude-3.7-sonne a hybrid reasoning model, scores above 20.0, indicating that it the only other model capable of constructing valid winning condition. In contrast, all non-reasoning models, including every open open-source small language model we tested, typically only managed to break the 'Wall Is Stop' rule, consistently earning score of 20.0. However, qualitative analysis of the models' self-defined subtasks and reasoning traces suggests that these models often fail to infer that breaking 'Wall Is Stop" is necessary prerequisite for constructing the winning condition. This implies that their success in breaking the rule was largely unintentional.
Table 42 presents ablation study evaluating the impact of agentic modules on performance in Baba Is You. Across both Llama-3.2-3B and GPT-4o, we observe no measurable improvement over the zero-shot baseline, with all configu guratiorsoons achieving the same score of 20.0. This indicates that the agentic components do not significantly enhance the agent's 's ability to reach the winning condition for these two models. The task remains challenging, largely due the model's limited spatial reasoning capabilities. While the models occasionally produce valid high-level plans, such as to form the rule 'Flag Win' at specific coordinates, they frequently fail to account for the game's pushing mechanics. As result, they often push text tiles in unintended directions, breaking misaligning the intended rule formation.
Table 43 presents ablation study input modalities across several multimodal models. We observe that relying solely on image input significantly degrades performance for all models. Adding image input top of text yields only marginal improvements, if any, suggesting that the agents primarily rely on text- based representations to make decisions. Notably, Gemini-2 2.5 -ro benefits slightly from the combined input, achieving the highest score of 86.7.
50

---

Published as a conference paper at ICLR 2026
N
2048
N.1 GAME DESCRIPTION FOR 2048
Environment. 2048 (Cirulli, 2014) is single-player sliding tile puzzle game played 4x4 grid. The objective is to combine numbered tiles by sliding them in one of four directions (i.e., up, down, left, or right) to create a tile with the value 2048. In this environment, the agent observes the current board state, represented as a 4x4 matrix of integers (each cell contains 0 for empty or a power of 2 for active tiles), and selects one of four discrete actions corresponding to directional moves. While the original game ends when no moves are available (i.e., the board is full and no adjacent tiles can be merged), we additionally terminate the episode if the agent performs five consecutive invalid moves (i.e., actions that result in no change to the board state). For implementation, we use an open-source, Pygame-based game environment. The 1 logic module manages the board state, tile movements, merging, and win/loss conditions, while the interface leverages Pygame to render the board and handle user input. The implementation supports dynamic resizing and contigurable parameters, and designed to facilitate both human play and automated experiments.
Observation-to-Ta Text Conversion. The environment's 's board state can be directly transformed into a textual description, formatting it as 4x4 array of integers in which each element indicates the value of the corresponding tile.
Action Space. The action space comprises four discrete actions: 'up'. 'down', 'left', and 'right', each representing a possible direction in which the agent can slide the tiles on the board.
N.2 GAMEPLAY PROMPT FOR 2048
Figure 24 shows the action inference prompt and the corresponding game screenshot used by the zero-shot agent to play 2048. The system prompt includes (1) the main objective of the game, (2) detailed game rules, and (3) the expected input-output format between the LLM and the environment. The user prompt provides (1) the specific task for the 2048 game, (2) the previous board state and game score, (3) the last executed action, (4) the current board state and game score, and (5) the expected output format. Based on this information, the agent determines the next action to take.
N.3 EVALUATION METRIC FOR 2048
The goal of 2048 is to create a tile with the value 2048. The game score increases as tiles are merged, with the value of the merged tile added to the total score. Although the score when the 2048 tile is created can slightly vary depending on the gameplay, it is generally estimated that the score is around 20,000 points. Therefore, define the evaluation metric the progress to the target score of 20,000, normalized to 100. Formally, the normalized score defined as:
Final Game Score
Score min
100, 100).
20,000
where 'Final Game Score' denotes the total score at the end of the game. This metric reflects how close the agent came achieving the primary objective of creating the 2048 tile.
N.4 EXPERIMENTAL CONFIGURATION FOR 2048
For all 6 open-source LLMs, including Llama-3.22 3.2-1-BB..1BBBB Dwen-2.553BB7B, and Minitron-4B/8B, we use a temperature of 0.0 and repetition penalty of 1.0. We the maximum number of game steps to 10,000. However, the maximum number of game steps (10,000) was never reached in the experiments. The game typically terminated either when the board was full and no adjacent tiles could be merged, when the agent failed to take an action that changed the board state for more than five consecutive steps. In the best gameplay episode of o3 zero-shot agent, which achieved a score of 57.32, a total of 685 steps were taken reach this result.
N.5 RESULT FOR 2048
All reported results in Tables 44-46 are computed as the mean and standard deviation over five independent runs, using 'zero-shot' agent for each model configuration. To establish baseline for
51

---

Published as a conference paper at ICLR 2026
System prompt
prompt
You un expertAl agent specialioed playing the 2045 zume with advanced scategg rezsoting.
Your primary schieve the lighot puoilic tile value while maintining lung-lemn
Merge
payahility by presersing flexhility board
Previous
2048 Riles
pene played grid. Take
'kowel; cnppty
ille with SAME value marg
[6,00 0)
2. 0, 0]
Marging directional:
[16, 4, 2,
merges
actions.
Soon: 52
-Cllnn-htuall mongos uccur
6. nies slide chowen direction as for as pasoibl, then merges applied
eet Last excecnted action
margo only ence per move
multiple somo-ralie liles aligndd (e D. 2,2,
l:ft
merges proceed flom the movenzer disection. For cuample:
12, 2, with scaalie in (4,
Curennt
-12, 2, with lesf sccalis in (4, 2, 2)
Board Gamex
5 An action only valid it csunes lrant one tile slide or margo Otherwine, the action
ignored, and new tile is spawaed
0,
7. After ever valid action,
til (issally 90 percest chance 2. 10 percent clance of appears
in madlsn
well.
2.
K gene ende when boord full volld posoble. value of the tile created
0 Soom incressos when
and the
You should unly ropood fimmst dosurieell below, you ustpat comment other information..
Desision
Provide your reporss the striet fororr
Analyse provided game
single optimal
Recsoning
your decision following Sornat:
<a detriled vammary of why this sation was choscs? 40bh Action.
Rossning
<cinectinn
detailed summary of why this oction cheseno
Actions
cup. right, left, dowas
screemshot
Emsure Batt
The Wel Ressonnnng field providos a clear explanation of why the sction is the hed choice, including analysis of current tlle positonss merge opportuillle, ind future fexubiliy. The pie Actiona' fiild untaire only one four valid dinccion.
2
4
4
16
4
2
Figure 24: Action inference prompt for 'zero-shot' agent playing 2048.
Models
2n48
Rank
Models
Agent
2048
Rank
Models
Input
2048
Rank
Random Agent
5.5e2
6
Random
5.5gr
Random Agent
Llama-3.2
0.0ea.
15
Zern-shor
0.3ge>
Text
S.6215
Qwen 2.5.7B
Qwen-2.5-30
JJama-3.2-38
0.3ea2
0.61a1
0.
12
13
11
Llama-3B
Reflection
Planning
Ref-Plan
0.0asa
0.1as2
11
GPT-4o
Imago
Both
Text
5.1
5.4gis
1,8
Minitron-4B
0.
14
Zero-shot
Gemini
Imuge
5,5
Minitron-BB
0.7ea>
10
GPT-4o
Rellection
Planning
3,5
6,0ass
Both
3,1
GPT-4o-mini
1.
9
Plan
Text Image
5.3
GPT-4o
5.6e1a
Claude
8.4
u3-mini
25.3
2
03-mini
Zero-she shot 25,3
Ref.
17.6ass
1
6.7gme
a4-mini
15.7
3
34.9
Gemini-2.5 2.ppro
5.1+22
8
Table 45: Ablation study for agen- modalities 2048. tic modules on 2048.
Table 46: Comparison across
Claude-3.7
Deepseek- RI
11,5
4
Table 44: Gameplay score on 2048.
comparison, we additionally included random agent that selects one of the four possible actions (up, down, left, right) uniformly at random. This agent was evaluated over 50 episodes, and its performance summarized in Tables.
2048. Beyond the 12 models presented in the main paper, also include results for OpenAl's more recent models, 04-mini and o3 in Table 44. Interestingly, none of the open-source models (e.g., Llama, Qwen, Minitron) were able correctly interpret the 2D array
Gameplay
52

---

Published as a conference paper at ICLR 2026
8
4
2
4
4
4
16
16
2
4
8
4
2
32
4
2
64
16
4
4
16
4
16
512
16
128
512
8
32
256
512
Figure 25: Three different runs for o3 zero-shot agent playing 2048.
prompt representing the game board. Consequently, they repeatedly issued the same action even when tiles could be merged, leading to premature termination of the game.
In contrast, GPT-40, Gemini-2.5 1.5inii-..5pra -pro, and Claude-3.7 were able to detect invalid moves and avoid them extent. However, these models failed manage merged tiles into their planning, often repeating superficially valid actions that quickly led to a board lock-up. As result, their gameplay performance was almost indistinguishable from that of the random agent.
mean- ingful gameplay performance. While the open-source and non-reasoning models were typically limited to producing maximum tile of 64 or 128, o3-mini successfully generated the 512 tile in of runs, and o3 achieved the 1024 tile in 2 out of runs.
Notably, the reasoning-capablee models-o3-mini 04-mini, o3, and DeepSeck-RI-exibee
A particularly interesting observation is that, even without explicit strategic instructions (e.g., corne- ing high-value tiles or arranging tiles staircase pattern), the reasoning- -based models implicitly discovered human-like strategies based solely on the basic game rules provided via system prompts. This behavior is illustrated Figure 25, where o3 exhibits an emergent form of spatial organization akin to that used by experienced human players.
Ablation study for agentic modules on 2048. In addition to models evaluated in the main paper, we conducted an extended ablation study using the 03-mini model to assess the impact of agentic modules-RRfflection and Planning-on gameplay performance in the 2048 environment. Table 45 presents the results of this study across three representative models: Llama-3.2-22 GPT-4o, and 03-mini, under various agent configurations.
For Llama-3.2-3B and GPT-4o, enabling the Reflection Planning together-die did not yield significant gains, suggesting that external agentic scaffolding alone is insuffi- cient for improving performance, likely due to limited reasoning architectural constraints.
In contrast, 03-mini exhibited a counter-intuitive trend: the zero-shot agent significantly outperformed both the Reflection and the Reflection- Peaaniing agents. plausible explanation for this phenomenon is that reasoning- centric models such as 03-mini may already perform internal reflection and planning when generating response. As such, the introduction of additional agentic modules may introduce redundancy or even propagate errors in intermediate reasoning steps, degrading performance.
Supporting this hypothesis, Figure 26 shows the inference output of o3 shortly before achieving 1024-tile. The output reveals that the model implicitly forecasts and sequences the next three actions, essentially performing internal planning without explicit agent modules. This emergent behavior highlights the potential of foundation models with strong reasoning capabilities to handle multi-step decision problems autonomously without external agentic augmentation.
Comparison across modalities on 2048. To assess the impact of input modality on gameplay performance, we evaluated models using either textual or visual (image-based) representations of the 2048 board state. The results are summarized in Table 46. Interestingly, GPT-4o demonstrated weaker performance with image inputs compared to text, suggesting that its image understanding capabilities least within the structured context of 2048 1448mmyylla lag behind its text comprehension. This is consistent with prior findings that GPT-4o, while multimodal, exhibits varying levels of alignment across modalities depending task complexity and structure. In contrast, both Gemini
53

---

Published as a conference paper at ICLR 2026
User prompt
Output (OpenAl o3)
Target Merge tiles to make a tile with the value of 2048
Reasoning
Straregic corner: Keeping the 512 in the bottom right corner is crucial stable Snukeemmontone board. **owwn"* move preserves this anchor, whereas "up"** would drag the 512 to the top zow and break the stracture.
#*A Previous state
Board of 2048 Games:
[2. 0, [2, 8, 0]
Immediate value: colurn the two 128 tiles merge into 256, giving instant hicrarchy
[8, 64, 64
and strengthening the bottom
[32, 128, 256, 512]
7312
(bottom becorne 32-256-256-5122)
executed action left
merges set up:
Row now contains two adjacent 256s that can merge into $12/1024 later right move.
#Hři Current
Row bacomes (2 0 Oj, easy subsequent left/ighht
Hoard 2048 Games:
swipe.
[4, 2,
[2, 8, 0]
Beard space: After the move the grid has empty cells ((4 0 0] [2 0 [8 0) [32 becathing room and reducing risk of dead-ead.
[8, 0,0]
512)), providing ample
[32. 128, 256, 512]
Score: 7444
Alternative moves:
You should only respord format described below, and you should output comments other information.
*"iight"* keeps corner but yields no merges and less space.
*Lfft invalid (no files move).
Provide your response in the strict format:
Up*** destroys the corner position and is therefore unsafe.
#ai Reasoning
<a detailed summary of why this action was chosen>
Given the merge gain, preserved corner, created future pairs, and increased empriness, **onn*** the optimal choice.
#A# Actions
-direction>>
Actions down
Figure 26: Output of OpenAl's A o3 model right before producing the 1024 tile in 2048.
Original systems prompt
Augmented system prompt
You an cauiigl player para Say Spiid Your colo daturaia bus daa harad
You sorve duak playse Saay The Spicai takkdd tf repiiaal o
thomat the noswerng fer when you abould do nou stop compieie the took 1heck you bht oupee the
Uffic copla unaien fer your cheven sap the granee, then spactyy proves sctivn
Resswiig dig dlokll tikik anp say and pooile consia sood big deta the and col eucurdd
Reswwing Asalyce say by rayy deliveing oa mwccd Secide aatageas out
Ralor
Game canage forloo
Due
foo, lip-
Ao sun of tum, union modiiiicu other citcits the player rectroc caecgr snd ditore cand-
tack ram hogare wish the player ktan cacrgy tnginng cash enlon sttordlly spectts officte
enygs
wayyy suppers
pereea
Moeaic will dumage ribce zacm comaate sack. 3 cadd cons ramusiing chuigyy walc playad, thai atfac vallo haad samurnt znarge
Maruzzu damags when ther mncoiive tachuly stts
Carda with
a sumpo otteca vacling scundiinn
Cven korta Aar
Wase you as CARD aar AND SELEC STATE, you cas shouss unc
Aakkes cuat sdanad eockatsdy Boee anallanle Actioss canade this allevee Hooll pense censs unoma fullis wilce madomriies varage ceaiially HlW END.
Mil dooca wik on frcettee given oili Any on watile seriictt fa
-i arhhre mabill actims, aqpurat sundus clanaars, 7LAY
ENO.
Figure 27: Example of original system prompt and augmented system prompt in Slay the Spire.
2.5 Pro and Claude 3.7 achieved better performance with image-based inputs than with text. This indicates a stronger visual reasoning capability in these models, particularly when parsing structured 2D spatial layouts such as the 2048 board. Their ability interpret and act visual patterns appears to be more robust than their capacity to process raw 2D arrays expressed as textual input, These findings highlight that multimodal models exhibit non-uniform modality strengths, and task-specific evaluations are crucial for selecting the appropriate input format to maximize agent performance.
O DETAILS FOR DATA AUGMENTATION
Prompting Details. To increase the diversity of our training dataset and avoid overfitting risks, we applied a data augmentation strategy focused on the system prompt. The original dataset was constructed by rolling out the same game prompt multiple times in the environment collect assistant responses. The system prompt remains static, and the user prompt exhibits variations due to changes in game state, though these were constrained to a fixed format where only specific values changed.
To address this issue, we performed augmentation on the system prompt, which contains general information such as the I.LM's role, game nules, and behavioral guidelines. Since the prompt was
54

---

Published as a conference paper at ICLR 2026
dependent on dynamic game states, modifying it risked introducing inconsistencies hallucinations. Therefore, we chose to keep the user prompt fixed during augmentation.
We use GPT-4o to generate paraphrased versions of the original system prompt. Specifically, prompted the model to produce 10 alternative phrasings the original system prompt that preserved its semantics while varying its linguistic expression. Each paraphrased system prompt then paired with the original user prompt and assistant response, resulting in 10 augmented versions of each original data point. Including the original, this expands the dataset by factor of 11. Figure 27 presents an example of the original and augmented prompt in Slay the Spire.
This augmentation strategy helped increase the syntactic and lexical diversity of the dataset while preserving semantic fidelity and coherence, thereby supporting more robust fine-tuning of the LLM.
# of Augs.
SuperMario
2048
24.2
1.4
26.8
3.5
10
26.7
2.8
20
25.9
2.4
Table 47: Performance across different numbers of augmentations.
Effect of The Number of Augmentations. Table 47 shows the fine-tuned model performance across different numbers of augmentations. We use the same fine-tuning setup as the intra-game generalization in Table 7. Overall, increasing the number of augmentations improves the performance of fine-tuned models compared to the one with no augmentation. Interestingly. the small number of augmentations (3 or 10) is quite effective to ensure linguistic diversity in fine-tuning, and the effectiveness diminishes as the number of augmentations grows.
P IMPLEMENTATION DETAILS
P.1 ASYNCHRONOUS INFERENCES FOR MULTI-AGENT ENVIRONMENT
Here we provide implementation details for multi-agent game environment that enables efficient, scalable, and realistic agent interaction.
The core design principle is to execute the game loop frame-by-frane basis while allowing each agent to asynchronously infer and initiate its next high-level action upon completion of the previous one. This approach is particularly well-suited for real-time, frame-based games such as Street Fighter III and StarCraft 11, where individual actions may span a variable number of frames (e.g.. 'Move Away' action takes 4 frames, whereas 'Super Attack' requires 7 frames complete). In this system, each agent operates independently, without waiting for other agents to complete their actions. As soon an agent completes its current action, it observes the current game state and determines its next high-level action. This asynchronous execution allows the game to progress fluidly and continuously, closely mimicking the dynamics of real-time multiplayer games.
Overview.
Illustrative Example. Consider a multi-agent scenario in Street Fighter III. Agent I selects 'Super Attack' that spans frames, while Agent 2 chooses 'Move Away' action that lasts 4 frames. The game orchestrator initiates both actions simultancously. After 4 frames, Player 2's action concludes, triggering the agent observe the updated state and select a new action, e.g., a 'Medium Punch' lasting 2 frames. The orchestrator then integrates this new action into the ongoing simulation, even as Agent I's "Super Attack" continues. This frame-by-fran Each agent re-enters the decision-makke making process immediately upon completing its current action, independent of the progress or status of other agents.
orchestration proceeds iteratively.
Advantages. This asynchronous, non-blocking scheduling model provides several key advantages. It enables agents to operate with varying action durations without artificial synchronization barriers, facilitating more natural and responsive interaction dynamic. The resulting system supports overlapping actions, better reflects the timing complexities of real-time games, and can be readily extended to support environments with more than agents.
55

---

Published as a conference paper at ICLR 2026
P.2 FINE-TUNING
We conducted supervised fine-tuning using collected gameplay data to adapt the L.M agent to game-specific reasoning and interactions with environments.
Task
LLMMA-3.--1Bh))
LLMMA-3.2-3B(h)
Intra- Matra-game/Non-game 00D-game
1.7
4.2
1.7
4.3
Table 48: Elapsed GPU time for fine-tuning.
Training Configuration. We fine-tune two models: Llama-3.22 1B and Llama-3.2-3B. Training is conducted using 4 NVIDIA A100 GPUs with 80GB of memory each. We use learning rate of 1e-6, a per-device batch size of with gradient accumulation steps set to 4, resulting an effective batch size of 64. The models were trained for epoch with 100 warm-up steps. Table 48 summarizes the GPU time taken for fine-tuning.
Data Statistics. In total, use 105,502 data points for fine-tuning. Only data points containing fewer than 4,096 tokens are used for training to ensure compatibility with model input length limitations. All data points from Pokémon and 329 out of 9900 data points from Minecraft discarded due to length constraint. For the sutoof-distribution (OOD) game generalization experiments, we exclude the games 2048 and Super Mario, resulting in 87,660 data points used. Apart from the number of data points, the training configuration remained identical.
P.3 UNSEEN SCENARIOS
To evaluate the intra- atra-gaame generalization capability of fine-tuned LLM agents, we define separate scenario that is not used during the training dataset collection. Unseen scenarios differs from the seen ones by featuring different character, map, or stage. Figure 28 presents example screenshots of the seen and unseen scenarios across six games.
Street Fighter III. While the character Ken was used during training data collection, different character, Chun-Li, was introduced at evaluation time to assess intra-game generalization. Chun-LL features distinct set of moves and hitboxes, posing significantly different control and tactical challenge compared to Ken. The game environment and match settings remained unchanged.
Darkest Dungeon. While the first expedition used during training data collection featured party composed of Plague Doctor, Vestal, H1 ghwayman, and Crusader, the unseen scenario involved different party composition: one Man-At- At-Arms, two Grave Robbers, and one Vestal. This change introduced significantly different combat dynamics, synergies, and positional requirements. Additionally, the enemy pool in this dungeon included the Ma Madma-aa monster not encountered during training maining-known known for its stress stress-inducing attacks and erratic behavior. All other gameplay parameters, including the dungeon type (short dungeon in Ruins area), remained unchanged.
StarCraft II. During training, agents were exposed to the 'Ancient Cistern LE' map, while evaluation was conducted on a different map, 'Babylon LE' to assess intra antra-game generalization. All other game settings, including player race (Protoss), opponent race (Zerg), opponent's build order strategy (Timing), and difficulty level (Hard), were kept constant.
Slay the Spire. We used game runs with different random seeds as unseen scenarios. Each game seed determines the layout of the map, including the sequence of opponents, bosses, events, and card reward options. All other game parameters, such as character choice (IronClad) and starting deck, were held constant.
Baba Is You. In the case Baba Is You, the second stage, Level used as unseen scenario. This stage features new rule combinations and object interactions that are absent from the training set, thereby testing the agent's 's ability to generalize to novel logic structures. The game mechanics and control scheme remained unchanged.
Now what is this?
56

---

Published as a conference paper at ICLR 2026
Q ANALYSIS OF THE GAP BETWEEN IN-GAME AND OOD-GAMES IN FINE-TUNING
Genre
Action
Adventure
RPG
Simulation
Strategy SlaySpire
Puzzle
Games
SuperMario
AceAttomey
DarkestD
MineCraft
2048
Llama-3.2-1B (Pretrain)
18.7
1.3
0.0
0.0
0.0
0.1
Llama-3.2-1 (Fine-tune)
23.5
5.5
0.0
0.0
0.0
0.3
Llama-3.2-3B (Pretrain)
47.5
0.0
0.0
Llama-3.2-3B (Fine-tune)
24.9
9.1
40.1
0.0
0.0
3.1
Avg. Improvement (%)
210.5
-7.8
0.0
0.0
1600.0
Table 49: Performance of LLaMA models with varying in-ood game gap.
Setup. For studying the in-OOD gap in generalization, fine-tune Llama-3.2 only Baba You. Other hyperparameter configurations are exactly the same as Appendix P.
Results. As shown in Table 49, fine-tuning yields the most improvements for OOD games in the same genre, but can also provide benefits across different genres in some cases. For example, 2048, another puzzle game, showed the largest improvement after fine-tuning. Interestingly, Ace Attorney, though different genre, also benefited from fine-tuning, likely due to shared requirements in logical reasoning (see qualitative examples below). In contrast, for games like Darkest Dungeon and Super Mario, which involve different gameplay dynamics and action spaces, the effect was mixed. Note that, as shown Table 7, unlike the current experiment (fine-tuned on Baba Is You only), LLMs fine-tuned on 10 games (without Super Mario) showed consistent improvements on Super Mario (OOD game). This implies our fine-tuning set, constructed from a diverse set games across genres, contains knowledge broadly beneficial for gameplay/decision-maak.. and supports cross-genre generalizability.
R UNIFIED PERFORMANCE COMPARISON
Table 50 compares the overall LLM/VLM agent results across agentic modules and modalities. Interestingly, one of the LLM agent that are using text-only states performs the best for each game.
S STATISTICAL SIGNIFICANCE ANALYSIS
This section reports three complementary statistical analyses: (1) per-game confidence intervals, (2) per-game significance tests using Welch's t-test (Welch, 1947), and (3) across-game paired I-tests (Fisher, 1925).
Per-Game Confidence Intervals. For each game, we report the sample mean ã, sample standard deviation s, the number of seeds n, and the two-sided 95% confidence interval half-width ("CI (95%)"" computed as
s
Cla5% = t0.975,
Vn
Table summarizes the results for GPT-4o.
Per-Game Significance Tests. We compare GPT-4o against 1lama-3.2-3B and Qwen-2.5-72B on each game using Welch's t-test (Welch, 1947). As shown in Table 52, GPT-4o and Llamma3.2233 shows significant deterministic differences in 10 out of 12 games, while GPT-4o and Qwen-2.5-72B shows significant or deterministic differences in out of 12 games. Here, "Deterministic" indicates both models have zero empirical variance and unequal means; "deterministic tie" indicates both zero variance and equal means.
Across-Game Paired Tests. To assess aggregate differences across games, conduct paired t-tests (Fisher, 1925) by pairing per-game means for each model. As shown in Table 53, GPT-4o significantly outperforms both 1ama-3.2-1B and Qwen-2.5-72BB
57

---

Published as a conference paper at ICLR 2026
Midel
Actiot
Advennur
RIG
Sinilation
Streayy
Pazole
SP3
Marer
Aoehmomey
HarSiery
Potémon
DarkestD
Mirecroft
Standew
StarCran2
StaySpire
BehalsYou
2048
Main Benchmark
Rosulis (Textonlyy Defaslt Agei)
L.am1-3.2-1B
0.0a0
0.0.00
0.0
0.030
0.0.aa
67anu
1Jama-13-90
13.321
M1
4.b115
4.211
0.0aso
47.5gm83
0.0ees
30.0
0.3se:
Qwen-2.5-50 2-53
43
1/2
0.0en
46.82731
0.0cm
0.0ems
0.aae
13.5mm
0.3
25-78
16.
27.2166
9,3
K511a
00:30
58.8128
0.0.na
0.0u00
0.1300
5.0.03
20.0.00
0.518-
Minitiz-488
167
4.bes
0.aan
0.0caa
0.fesn
20.0ma
0.1sa0
Minitoz 38
23.3.14
51,31135
299136
K2:11
00/30
638
0.00a
00130
0.0.03
20.0.03
0.7147
33,3
12.0a:14
53
46.4-44
187ama
$7.3acs
0.0ass
40.9mm
0.
5.0mra
30.0-z
Uwen25:77
2677
299987
410011
27N
18M22677
01/30
9:0-33
20,0.0
GPM--oomiii
167
28.3
M5
21.3asa
0.
$1.3asa
46.0m1
16 1elene
15.0emm
13,5m
GPT-40
29.7_3-3
54.11143
853111
64.2122
189.99
93.4115
71,011
S14148
108.0.ca
2361221
20.0.00
5,615
GPT5
185-110
31.62065
99. 1131
749
389
92
73,0
92.3166
25.0
26.22184
100...3
10 10:2121
ad-miii
33.3-113
34.
01.7
66.5p3
0.0.0
59.0.c:
75.0
25.0uamo
150cco
73.3
25. 15
Genme-2.5 5-pm
133-311
577634
07:335
35.3n0
957.11
75:0
S1.
73
167-11
31.7182
810
02.934
63.0
59.0.3
75.0
53
500.338
15,0ua
467744
533
Deopeock-R1
Agentic Midule Ablation 3BB.3
GPT)40)
1.MMA-18 (Zamnehen)
21-3aa
5.7ss
422r
0.0mm
47. 47.5
0.0am
0.0caa
0.Bes
20.0
0:3ea:
LLaMA 18 (Retlection
300
52.
4.5.13
444113
0.0300
47.5,200
0.0.08
0.0000
00130
0.0.03
20.0.03
0.010
1.aMMA-1B (Panning)
30.0.s
27.0aa
4.50sa
$2ar8
0.0ss
0.0aus
0.0eaa
0.
0.0as
30.0-a
LLMMA-78 Mont
1677.3
31
3800
54404
00:300
5T00
0.una
01130
00000
20.0.00
0.31e:
GiPE-l (Zenosher)
20/6::
49
6423
333eem
03
14.5%
347257
GPT-4a (Reflecion)
23,
52.31155
853115
01.5144
26.1
15
50,0
15,5145
0.1300
41,31188
20.0.00
3.5129
GPT-4o (Planning)
30.0334
2944115
5277
594117
3333500
13,0u00
54M2302
50.0
35,3
20.0.00
6:115
GPT-4o (Rol-Paan)
23.3.30
34.1.442
$2.8
60.0c49
38.9
91
50.0ses
R1.4
108.
368.ps
20.0aas
7/0asc
Medulity AMoriinn [Test/ Emogee Both)
Uven25.V S-L-7 (Test)
0Mlano
260283
100
2.8
W.Mons
0.0caa
0.eona
13.an
0.1
Qwen? 5-V1-78 (imagr)
0.7ses
Quee2 VL (Both)
33158
25.6100
1761121
2550
2.144
$18138
0,000
0.0100
0.0.03
0.0.30
0,4185
Uwer2 s V1-318 (lext)
167:11
Manss
154
278
39M215
1522227
0.1300
20.0.0
0.2aes
Qwes2 S.V 32B (mssge)
33.3-3
13.3
(lorh)
15
UPT-4 (Test)
25.7.143
S012:41
N53
64.2122
38Me
934811
R1.414
1980:ca
2M
20,0.0
GPT-4 (Emagc)
23.7
27.1
0.0mea
6 67
GPT-4o (Both)
34,32143
27.1
53.537
01
41 41.7253
92.7
41 Mam1
50.001t
23
20.0en
5/444
Gemin-2.5 (Test)
13.3mm
38.0..a
53.7as
67.5
$3.3
9377
75.0
19.2111
1080.c
51.9
73,3
(image)
16:7-113
Gemini-2.5
$2.6axx
$3.3
92.2a1a
100.0.c
26.2.m
867asii
Chasio-33 (Tox)
167en
18 28.2
81.9et
62.0pt
63 Peses
$9.9
75,0
63
50.0escx
15,0ca
46, 66.7o
5.3
Cluice 3.7 (Image)
23.3.114
9,0una
20.0.00
8.4100
Chasde-3.7 (Bolh)
33.3.a
22
71 1
63,6
22
90.1 Lasa
40.5 40.5pmu
30.0
9.7
20.0aus
6.7
Homon Novice
Humas (Novioe)
20.0.3w
1000
87.8
36.1
90.3.2a
70.8.0
33,3.00
52.9222
1000.4>
Table 50: Unified LLM/VLM results of Orak: (Top) Main performance (all models), (Middle) Agentic module ablation (LLaMA-3B / GPT-4o), (Bottom) Modality ablation (Tex//mnac//ooth) The best values for each game are marked in bold, and missing Image results are shown as
Model
Metric
SF3
SuperMario AccAttorney HerStory Pokéimon DarkestD Minecraft Stardew StarCraft2 SlaySpire BahalsYou 2048
# seed
20
3
3.
3
3
3
3
moan
29.7
34.1
85.3
64.2
38.9
93.4
71.0
81.4
100.0
23.6
20,0
5.6
GPT-40
std
14.3
14.2
1.5
5.2
9.6
1.5
7.0
4.8
0.0
22.1
0.0
1.5
CI (95%)
13.96
6.65
3.73
12.92
23.85
3.73
17.39
11.92
0.00
54.90
0.00
1.86
Table 51: Per-game statistics for GPT-4o. "CI (95%)) is the two-sided confidence interval.
T DISCUSSION FOR INTEGRATING ORAK WITH MULTI-TURN RL FINE-TUNING FRAMEWORKS
Orak can be readily integrated with existing multi-turn RL fine-tuning frameworks for LLM agents (Wang et al., 2025b; Zeng et al., 2025). In based environments, rewards are often sparse, being provided only at the end of an episode (e.g.. final score). Therefore, careful reward design and dynamic data extraction are crucial for stable and effective optimization.
Reward Design. We discuss two types of reward designs.
(1) Reward Discounting. Following common practices in reinforcement learning, we apply reward discounting to estimate intermediate rewards during gameplay:
R T- SSinal
(2)
where 0 < < < 1 denotes the discount factor and Sjial is the final game score.
(2) Turn- Rewards (when available). For games that provide intermediate feedback, such as 2048 or Super Mario, a turn-level reward can be incorporated as:
R₂ = XS + (1 >)> Th SSiaal
(3)
58

---

Published as a conference paper at ICLR 2026
Game
bstaat
p-value
Interpretation
Game
E-stat
p-value
Interpretation
SF3
4,644
0.00997
significant
SF3
0.435
0.681
not significant
Super Mario
4.149
0,0002
sägnificant
Super Mario
1.128
0.268
dgnificant
Ace Atlomey
54,641
0.0001
significant
Ase Allemey
14.127
0.0039
siguificntt
Her Story
20.155
0.0015
significant
Her Story
7.345
0.0104
significant
Pokémon
7,018
0,0197
significant
Pokémon
1.416
0.230
not significant
Darkest Dungeon
107 849
0.0001
significant
Darkest Dungeon
significant
Minecraft
568
0,0032
significant
Mineeratt
17-568
0.0032
significant
Stardew Valley
29.373
0.0012
significant
Stardew Valley
6.320
0,0164
significant
StarCraft II
deterministic
StarCraft II
deterministic
Slay the Spire
1.850
0.2056
not significant
Slay the Spire
1.113
0.371
not significant deterministic tie significant
Baba You
2.003
0,1831
not significant
Baba Is You
2048
8,329
0.0011
significant
0.0013
(a) GPT-40 vs. Llama- 3.2-3B (Welch's 's t-test).
(b) GPT-4o vs. Qwen-2.5-72B (Welch's t-test),
Table 52: Per- Per-game significance tests (Welch's 's t-test).
Comparison
t-stat
p-value
Interpretation
GPT-4o vs. Llama-3.2-IB
5.227
0.00028
significant significant
GPT-4o vs. Qwen-2.5-72B
3.122
0.00971
Table 53: Cross-game significance tests (paired t-test).
where λ is balancing coefficient and S, the turn-level score. This formulation allows smoother credit assignment across gameplay turns.
Dynamic Data Extraction. During fine-tuning, the model repeatedly performs data rollouts, reward estimation, and multi-turn RL optimization across multiple episodes. As the policy improves, it naturally encounters increasingly challenging game scenarios, promoting progressive data extraction and richer experience replay. After each episode, validation accuracy is compared between the current and previous policy models, and the superior one is retained. This iterative selection procedure contributes to stable and robust policy improvement over time.
U LIMITATIONS AND FUTURE WORK
Cost Consideration. Among all games Orak, six games, i.e., Ace Attorney, Her Story, Darkest Dungeon, Stardew Valley, Slay the Spire, and Baba is You, require one-time purchase, typically priced ranging from $9.99 to $24.99. While this represents a non-negligible upfront cost, it is relatively minor compared to the recurring cost associated with proprietary LLM API calls. From cost-efficiency perspective, the benchmark remains accessible and practical for sustained research.
License Issue. We have made considerable efforts comply with licensing requirements in designing our benchmark. (1) Users must purchase the commercial games themselves for evaluation. (2) We do not distribute any commereial game executables. (3) We do not modify any game assets during gameplay. Moreover, we explicitly state the following legal compliance guidelines "This project is strictly for research purposes. Researchers are required use our framework solely as a benchmark for evaluating LLMs, and any models derived from our benchmark are strictly prohibited from commercial use".
Real-time Gameplay. Street Fighter III. Super Mario, and StarCraft I/ inherently require real-time gaming, unlike other turn-based or simulation games in Orak. However, in our current evaluation setup, the game is paused during L.M inference to remove the impact of real-time constraints on agent performance. While this allows for more stable evaluation of reasoning capabilities, real-time responsiveness is critical in many gaming contexts, so it should be handled for practical needs. We leave latency-aware evaluation protocols for building real-time gaming LLM agents as future work.
Study on RL-based Fine-tuning. Although RL-based fine-tuning has demonstrated strong perfor- mance in many domains, such as mathematics and programming, we did not explore it in this study. Given the interactive nature of the Orak environments. it would be natural to derive dynamic, context- aware rewards from in- in-game feedback and apply RL fine-tuning methods such as DPO (Rafailov et al., 2023) GRPO (Shao et al., 2024). Unlike domains such mathematics or programming,
59

---

Published as a conference paper at ICLR 2026
where problems typically have static correct solutions derived through logical reasoning, gameplay requires strategic reasoning that adapts dynamically to the actions of other agents. In games, the optimal action is often contingent the evolving behavior of the player opponents, reflecting the complex, interactive nature of multi-agent environments. This distinction is particularly relevant to real-world domains such as business, economics, and negotiation, where strategic decision-making is frequently modeled using game-theoretic frameworks (Zhang et al., 2024). Therefore, leveraging Orak as an environment for RL-based fine-tuning may significantly enhance an LLM's capacity to reason strategically and operate effectively in multi-agent settings. This line of research holds promise for improving LLM performance in broad range of real- aal-world applications where understanding multi-agent dynamics is essential.
Support for Diverse Modalities. Orak supports evaluation of LLMS and VLMs, but does not extend to other modalities often essential in real-world gameplay. One key example is sound. For instance, in Minecraft, players rely on zombie audio cues to avoid danger, while in Street Fighter I!I and StarCraft II, alert sounds indicate some specific attacks by the enemy. Also, beyond games in Orak, First-Person Shooter (FPS) games usually use gunfire sounds for spatial awareness, and horror games often rely audio signal the proximity of threats. Consequently, incorporating other modalities such as audio remains an open challenge, and benchmarking the performance of emerging LanguugL Models (Chu al., 2023; Cui al., 2024) in gaming scenarios could be an important step toward broader multimodal agent development.
Societal Impact. Simulation games, Minecraft and Stardew Valley, offer rich environments where players can explore, mine resources, and craft items, enabling life-like simulations of human behavior. These games offer valuable testbed for analyzing long-horizon behavior of LLM agents by sys- tematically comparing gameplay trajectories of humans and LLM agents, which enables a rigorous assessment of whether LLMs exhibit human-like decision- decision-making patterns. Introducing multiple agents into these environments allows for the study of emergent social behaviors among LLM agents. We believe such settings are particularly well-suited for precisely measuring the social impact of complex agent behaviors, offering valuable insights into the dynamics of M----bseed agents.
60

---

Published as a conference paper at ICLR 2026
86
1:000
(a) Street Fighter III
(b) Super Mario
(c) Darkest Dungeon
Bodes
(d) Starcraft 11
Leguee
Engrenal
Thile
Ticle
Trxsarn
(e) Slay the Spire
18
MC
IS
(f) Baba Is You
Figure 28: Comparison of seen (left) and unseen (right) scenarios for six games.
61


Reviews:
[{"review_id": "b7QkQicpx6", "reviewer": "Reviewer_mcXu", "summary": "The paper proposes Orak, a benchmark and dataset for evaluating foundation models in dynamic digital games scenarios. Orak increases the diversity of game genres covered in evaluation when compared to previous benchmarks, and the developed platform also allows plug-in and enabling/disable different agentic modules for ease of evaluation/ablation. \n\nMoreover, Orak includes a dataset of fine-tuning data, collected as interaction trajectories generated by foundation models guiding playback of all 12 games supported in the platform.\n\nExperimental results show the performance of 15 foundation models on the benchmark, an ablation study of 2 of those models utilizing different agentic approaches to play all games, results on scenarios combining different data modalities, and SFT results using Llama to illustrate the benefits of the collected trajectory dataset.", "questions": "I don't think the games industry itself is the main beneficiary of such benchmarking effort. How does this motivation and focus affect the benchmark design? Agent autonomy using games as learning environment could have much wider impacts and would need to be analysed differently.\n\nThe paper mentions previous evaluation \"often rely on visual inputs\" as a criticism. Why? I'd argue that understanding of visual observations is exactly the most important area where games can help as benchmark for foudnaiton model capabilties.\n\nIn the Arena setting results, why exactly was Starcraft evaluated with one less model than Street Fighter? \n\nAlso, critically, do you have any insights on why Minitron-8B performs so much better than in full results? Minitron was 0 for Starcraft in Table 3. Any deeper understanding here could be significant.\n\nBTW, Table 3 shows the results only of using the \"auto extracted state in text only form\", correct? It would be interesting to have an easy comparison of that vs. best resutls somewhere. Even if in an appendix.\n\nIn a similar veing to the Arena question for Minitron, do you have any insights on why there is no SFT benefit intra-game for Startcrat and Baba?\n\nTypo:\n\"In game industry\" -> \"In the games industry\"", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Orak provides a very funcional benchmark platform for different game genres, as well as a somewhat general covered of game genres that allows better insights about required capabilities from foundation models during gameplay. Such results can also potentially generalize to wider impact beyong digital games alone.\n\nThe presented results well illustrate the current capabilities of foundation models and how different games probe them in different dimensions.\n\nThe created dataset (and, more importantly, data collection platform) can also serve as a stepping stone for further research on improving gameplay models/agents/systems.", "weaknesses": "While well presented and illustrating the potential of the benchmarl in principle, the current paper presents some limitations. Mostly regarding analysis of the results and the rationale for some of its design aspects.\n\nThe manuscript claims \"in-depth analyses of input modality, agentic strategies, and fine-tuning effects\", but falls a bit short of this challenging goal. It does provide some interesting insights, but doesn't really go deep into either of the 3 areas. Having said that, the platform itself is already of great value and can be used as a strong base for further evaluation and analysis. Toning down such claims still leaves the rest of Orak as solid work.\n \nRegarding input modalities, Orak emphasizes pre-extraction of game state information and textual representation of such data. This both; i) greatly reduces the challenges of observation/state/world understanding and already bias results to text; and ii) muddles the analysis of providing different modalities later which then also needs to deal with potential conflicts in different modalities and in some models seemingly having preference for specific modalities. Also known issues previously discussed in the literature.\n\nThe presented ablation for agentic modules is interesting as a high-level overview, but its results as presented are not well discussed and don't show significant insights. The analysis here could go much more in-depth in future work.  \n\nThe LLM finetuning experiment and resutls are not in-depth and only take a quick look at 1 foundation model in 2 small sizes, where the benefits of SFT with any data would already provide the most benefit. The paper would benefit from a more detailed analysis of dataset quality and what/how it actually contributes during training, as well as any possible insights into data scalling.\n\nHowever, the main limitation of Orak as a platform is the lack of functionality to properly evaluate grounding of actions, as each game action space has been manualy defined differently and already mapped to high-level functions that heavily abstract and simplify the problem. Though, the platform could be easily modified to offer a full pixel-to-keyboard/mouse-commands interface that fully exposes complexity and allow uses to benchmark at different levels.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LRQIYlv7Ha", "reviewer": "Reviewer_Cdp1", "summary": "This paper introduces Orak, a benchmark for evaluating LLM agents across 12 video games spanning six genres. The authors aim to address limitations in existing game benchmarks by offering greater diversity, enabling studies on agentic modules (such as reflection and planning), and providing resources for adapting LLMs into gaming agents. The key contributions are the benchmark itself, which uses a plug-and-play interface based on the Model Context Protocol (MCP) for standardized evaluation, and a fine-tuning dataset derived from expert LLM trajectories designed to distill gaming skills into smaller models. The paper presents a series of experiments on 15 LLMs, analyzing their performance, the impact of agentic modules, the effect of visual inputs, and the generalization capabilities of fine-tuned models.", "questions": "The paper positions Orak as a foundational benchmark that pushes the boundaries of agent evaluation. However, the tasks often seem simplified through pre-processed states and high-level APIs, which might not establish a clear differentiation from prior work.\n\nCould the authors elaborate on the unique challenges Orak presents compared to existing agent benchmarks? The current results do not seem to establish a clear differentiation, as the main conclusions are largely echoes of findings from other domains.\nWhat specific agent capabilities are uniquely tested in Orak that are not adequately covered by prior benchmarks? A more compelling case could be made by showcasing a task where top-performing agents from other domains systematically fail due to a game-specific challenge that Orak is specifically designed to evaluate. For instance, is there a scenario that rigorously tests an agent's ability to reason under partial observability from raw signals, a core challenge in many games?\n\nI'm willing to increase my score if the author can answer my question.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The primary strength of this paper is the scale and diversity of the benchmark. Compiling 12 games across six distinct genres, each with its own environment setup and state representation, is a big **engineering effort.** This provides a broad testbed for evaluating a variety of agent capabilities, from reaction time in action games to long-term planning in strategy games.\n\nThe introduction of a unified, plug-and-play interface using MCP is a commendable step towards standardized and reproducible evaluation of LLM agents in gaming environments. \n\nThe release of a fine-tuning dataset, while based on LLM-generated trajectories.", "weaknesses": "The paper, despite its significant engineering effort, suffers from several weaknesses in its core claims, methodology, and the novelty of its conclusions, which limit its overall contribution.\n\nLargely Unsurprising and Incremental Conclusions: The main findings drawn from the extensive experiments largely confirm well-established knowledge in the LLM agent community, offering little new insight.\n\n- The conclusion that proprietary, closed-source models outperform their open-source counterparts is widely accepted and requires little further validation in 2025.\n\n- The finding that agentic workflows (e.g., reflection, planning) benefit capable models is not new.\n\nThe claim that visual inputs often hinder performance is misleading. This outcome is likely an artifact of the experimental design, where highly structured and pre-processed text provides a cleaner, more direct signal than raw visual data for current VLMs. A more accurate conclusion would be that under this specific setup, the models fail to extract sufficient value from visual inputs to overcome the noise, rather than a general indictment of visual modalities for gaming agents.\n\n\nQuestionable Design Choices in Benchmark and Data Generation:\n\n- The selection of games appears to be driven more by the availability of existing APIs or emulators (e.g., Mineflayer for Minecraft, PyBoy for Pokémon Red) rather than a principled selection of titles that would best probe the frontiers of AI capabilities. The benchmark lacks modern, complex 3D games that pose severe challenges in perception from raw pixels, physics-based interaction, and complex spatial reasoning.\n\n- The fine-tuning dataset is generated by an 'expert' LLM (GPT-4o), which fundamentally caps the potential performance of any fine-tuned model at the level of the teacher model. This methodology prevents the discovery of novel strategies that might surpass the teacher's capabilities and introduces the teacher's inherent biases and failure modes into the student models.\n\n- By selecting only the highest-scoring trajectories for the fine-tuning dataset, the authors introduce a strong survivorship bias. The models learn from 'perfect' or near-perfect executions ('sunny day' scenarios) but are not exposed to data on how to recover from mistakes, adapt to unexpected situations, or turn a losing game around. This is a critical omission for developing robust agents that can handle the stochasticity and adversity inherent in complex games.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ygzOPCe4TX", "reviewer": "Reviewer_Tpyv", "summary": "This paper proposes **Orak** benchmark and evaluation suite for training and evaluating LLM/VLM agents on a diverse set of real video games. The benchmark includes 12 games spanning action, adventure, strategy, and puzzle genres, a modular agent interface (reflection / planning / action / memory components) built on a Model Context Protocol (MCP), prompt templates and action-space definitions, and an expert-trajectory dataset used for supervised fine-tuning. The submission reports cross-model comparisons, ablations of agentic modules, and fine-tuning experiments including transfer tests to out-of-distribution and non-game tasks.", "questions": "1. For each major table/figure, how many independent trials and random seeds were used? Please add confidence intervals and describe any hypothesis tests performed. If trials vary by game, report that explicitly.\n2. Did you run controlled sweeps over prompt phrasings, temperatures, context lengths, or token limits when comparing agentic modules? If not, please run such sweeps for key games or explain why module effects are independent of prompt variants.\n3. Can you provide at least one latency-aware experiment for a timing-sensitive game (e.g., impose an upper bound on LLM response time or simulate delay) and report performance degradation as a function of latency?\n4. Fine-tuning data & overfitting controls. For the supervised fine-tuning dataset: how were trajectories sampled (top trajectories or diverse sampling)? What regularization, early-stopping, or validation protocols prevented overfitting to a specific agentic workflow?\n5. I request ablation results comparing fine-tuning on different generator sources (e.g., gpt4o-only, o3-mini-only, mixed-source, and, if available, human or RL trajectories) to quantify generator-specific biases.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. **Broad coverage & engineering effort.** The benchmark covers 12 real games across diverse genres and provides a modular evaluation harness, which is useful for benchmarking different LLM agent designs.\n2. **Multi-dimensional ability taxonomy.** The paper defines and uses a set of capabilities (e.g., long-horizon planning, spatial reasoning, rule compliance) and maps games to these capability needs, enabling capability–task analyses.\n3. **Publicly available supervised trajectories & fine-tuning experiments.** The authors collected and provide a dataset of expert LLM interaction trajectories and demonstrate supervised fine-tuning improvements and some transfer effects.\n4. **Useful baseline comparisons.** Results compare multiple closed-source and open-source LLMs under several agentic strategies (zero-shot, reflection, planning, ref-plan), giving a practical snapshot of current model gaps and engineering trade-offs.", "weaknesses": "1. **Insufficient novelty argument / differentiation from prior benchmarks.** The paper lists related benchmarks but does not convincingly quantify or empirically demonstrate how Orak meaningfully advances beyond existing game/agent benchmarks. The unique scientific questions Orak enables are not sharply distinguished.\n2. **Experimental robustness & statistical reporting are incomplete.** Many reported results lack rigorous statistical detail (consistent number of seeds/trials, confidence intervals, or significance testing). Some reported scores show large variance, which weakens the reliability of the conclusions.\n3. **Lack of systematic prompt / hyperparameter sensitivity analyses.** The paper attributes improvements to agentic modules (reflection/planning), but does not systematically vary prompt wording, temperature, max-context length, or other prompt-engineering factors to rule out that effects are largely prompt-driven.\n4. **Real-time interaction and latency issues under-addressed.** For latency-sensitive or action-timed games (e.g., fighting or platformers), the evaluation often pauses the game during inference. This departs substantially from real-world online agent constraints; latency-aware experiments are deferred to future work, limiting external validity.\n5. GPT-generated trajectories (e.g., gpt4o, o3-mini) **risk bias, hallucinations, low strategy diversity, and contamination**; the authors should document generation details and show that fine-tuned models generalize beyond the generator.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mPcMHm7bTo", "reviewer": "Reviewer_5244", "summary": "This paper presents ORAK, a comprehensive benchmark designed to evaluate and train large language model (LLM) agents across a diverse set of 12 video games spanning six major genres. The benchmark addresses critical gaps in existing game-based evaluations by incorporating complex, real-world video games, providing modular agentic strategies (e.g., reflection, planning, and tool use), and releasing a high-quality fine-tuning dataset derived from expert gameplay. The authors also propose a unified evaluation framework that includes leaderboards, LLM battle arenas, and in-depth analyses of input modalities, agentic strategies, and fine-tuning effects. The experiments demonstrate the capabilities and limitations of both proprietary and open-source LLMs across various tasks, showcasing the potential of ORAK as a foundation for advancing general-purpose gaming agents.", "questions": "See weakness.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "To be honest, I'm very excited to see an LLM benchmark that integrates complex games. The LLM field is currently flooded with a large number of fixed datasets, yet claiming to be “agentic” is clearly insufficient. ORAK covers a wide range of game genres, including action, adventure, role-playing, simulation, strategy, and puzzle games. This breadth ensures a holistic evaluation of LLM capabilities, from logical reasoning to spatial understanding and long-term planning. The use of the Model Context Protocol (MCP) to integrate LLMs with game environments and agentic modules is a significant contribution. This modular approach enables plug-and-play experimentation and facilitates systematic studies of LLM behaviors in diverse scenarios. The release of expert gameplay trajectories across multiple genres is a valuable resource for the community. The dataset encapsulates meta-knowledge and demonstrates how fine-tuning can enhance LLM performance in both gaming and non-gaming tasks. The paper provides extensive experimental results, comparing proprietary and open-source LLMs across tasks and modalities. The insights into the effects of fine-tuning, visual inputs, and agentic strategies are particularly compelling.\n\nI strongly recommend acceptance of this paper. It makes a significant contribution to the field of LLM evaluation and training, offering a robust benchmark that bridges the gap between academic research and real-world applications. The paper is well-written, methodologically sound, and forward-looking, providing a solid foundation for future work in gaming AI and general-purpose LLMs. I particularly appreciate the authors' attention to detail in designing ORAK and their commitment to open science through the release of datasets and tools.\n\nThis work is not only timely but also highly impactful, and I believe it will become a key reference for researchers in the field.", "weaknesses": "This paper still has some minor flaws, and I hope the authors will pay attention to the following issues.\n\n1. The current setup pauses games during LLM inference, which simplifies evaluation but does not fully reflect real-world gaming scenarios. Including preliminary results or discussions on latency-aware evaluation protocols would strengthen the paper.\n\n2. Although the paper mentions RL-based fine-tuning as future work, a brief discussion on how ORAK could be adapted for RL experiments (e.g., reward design, dynamic data extraction) would be valuable for readers interested in this direction.\n\n3. The authors acknowledge the cost of proprietary games and LLM APIs. Exploring potential solutions, such as open-source alternatives or simplified game environments, could help lower the barrier to entry for researchers with limited resources.\n\nIf the authors could provide detailed data for problems 1 and 2, as well as potential solutions for problem 3, I believe this paper would be worthy of a spotlight paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)