PokéChamp: an Expert-level Minimax Language Agent
Authors: Seth Karten
Andy Luu Nguyen
Chi Jin
Venue: ICML 2025 spotlightposter (2025)
Keywords: multi-agent systems
LLM Agents
competitive games
partially observable
test-time compute
pokemon
Primary area: reinforcement_learning->multiagent
TL;DR: PokéChamp, an LLM-powered AI agent for Pokémon battles, outperforms existing bots and achieves expert-level performance against human players using minimax search with LLM-based action sampling, opponent modeling, and value calculation.

Abstract: We introduce PokéChamp, a minimax agent powered by Large Language Models (LLMs) for Pokémon battles. Built on a general framework for two-player competitive games, PokéChamp leverages the generalist capabilities of LLMs to enhance minimax tree search. Specifically, LLMs replace three key modules: (1) player action sampling, (2) opponent modeling, and (3) value function estimation, enabling the agent to effectively utilize gameplay history and human knowledge to reduce the search space and address partial observability. Notably, our framework requires no additional LLM training. We evaluate PokéChamp in the popular Gen 9 OU format. When powered by GPT-4o, it achieves a win rate of 76\% against the best existing LLM-based bot and 84\% against the strongest rule-based bot, demonstrating its superior performance. Even with an open-source 8-billion-parameter Llama 3.1 model, PokéChamp consistently outperforms the previous best LLM-based bot, Pokéllmon powered by GPT-4o, with a 64\% win rate. PokéChamp attains a projected Elo of 1300-1500 on the Pokémon Showdown online ladder, placing it among the top 30\%-10\% of human players. In addition, this work compiles the largest real-player Pokémon battle dataset, featuring over 3 million games, including more than 500k high-Elo matches. Based on this dataset, we establish a series of battle benchmarks and puzzles to evaluate specific battling skills. We further provide key updates to the local game engine. This work establishes Pokémon as a benchmark to integrate LLM technologies with game-theoretic algorithms addressing general multi-agent problems. Videos, code, and dataset are available online.

PokéChamp: an Expert-level Minimax Language Agent
Seth Karten Andy Luu Nguyen
Chi Jin'
Abstract
100
We introduce PokéChamp, minimax agent powered by Large Language Models (LLMs) for Pokémon battles. Built on general framework for two-plat -wo-player competitive games, P PokéChamp leverages the generalist capabilities of LLMs to enhance minimax tree search. Specifically, LLMs replace three key modules: (1) player action sam- pling, (2) opponent modeling, and (3) value func- tion estimation, enabling the agent to effectively utilize gameplay history and human knowledge to reduce the search space and address partial observ- ability. Notably, our framework requires no addi- tional LLM training. We evaluate P PokéChamp in the popular Gen OU format. When powered by GPT-4o. achieves win rate of 76% against the best existing L.LM-based bot and 84% against the strongest rule-based bot, demonstrating its su- perior performance. Even with an open-source 8- Aillion-paramedd Llama 3.1 model, P PokéChamp consistently outperforms the previous best LLM- based bot, Pokéllmon powered by GPT-4o, with 64% win rate. PokéChamp attains projected Elo of 1300-1500 online ladder, placing it among the top 30%- 10% of human players. In addition, this work compiles the largest real-player Pokémon battle dataset, fea- turing over million games, including more than 500k high-Elo matches. Based on this dataset, we establish a series of battle benchmarks and puzzles evaluate specific battling skills. We fur- ther provide key updates to the local game engine. This work establishes Pokémon as benchmark to integrate LLM technologies with game-theoe theoretic algorithms addressing general multi-agent prob- lems. Videos, code, and datasct are available on- line.
Flyer Percente (N)
80
Key: Bot, Eio
@itnk 1990
Gltips 1300-1502
#rotaChhrmp;
@rstedLMer; 1303-1370
Whing Pl ayer: 1200
@kbyssst 1002-1180
@tanndmm 1000-1005
Rati
Figure 1. PokéChamp achieves the 70%-90% percentile of play- and 1300 Elo rating against real players. Higher Elo and percentile denote better performance.
1. Introduction
Realizing superintelligencee "takeoff" demands cycle of recursive self-improvement in which each model itera- tion can outplay the last-a capacity currently bottlenecked by the strategic search and coordination challenges we ob- serve in competitive games. Prior work in reinforcement learning has achieved significant success, attaining super- human performance in wide range of games, including Chess, Go, and Poker, through extensive imitation learning and self-play (Campbell et al., 2002: Silver et al., 2017; 2016; Brown & Sandholm, 2019; 2018: Vinyals et al., 2019; Berner et al., 2019). However, these approaches typically require substantial task-specific training and engineering. In contrast, Large Language Models (LLMs), which often function generalist agents, have demonstrated remarkable capabilities across various domains. Language agents leverage prior knowledge of game strategies and generalize to new situations without additional training.
the Pokémon Showdown
Despite their promise, recent studies highlight limitations in the planning capabilities of text-based language agents (Topsakal & Harper, 2024). These agents frequently un- derperform compared rule-based heuristic bots in game environments (Küttler et al., 2020) and struggle grasp ba- sic game mechanics (Hu et al., 2024b). To overcome these challenges and further investigate the potential of LLMs in complex game environments, this paper focuses on a fast paced, strategy-rich. and challenging two-player com- petitive game game-Polees -ookéoon battles.
'qqual contribution 'Piinceton University. Correspondence to: Seth Karten esethkarten princeton.edu>.
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

PokéChamp: an Expert-level Minimax Language Agent
Pokémon battles present a unique and formidable challenge for AI systems. With over 1000 Pokémon species, each pos- sessing unique abilities, moves, typing, and stats, the state space complexity is estimated to be on the order of 10354 for the first turn alone (The-Thiree ThirddBBuid, 2022). The game features partial observability, where information about the opponent's 's team is gradually revealed through gameplay, maintains this vast search space throughout the battle. Fur- thermore, Pokémon battles can last anywhere from 6 to over 100 turns, making exhaustive tree search over all possible branches computationally intractable.
ous competitive scenarios. Our evaluation includes arena- style comparisons against established Pokémon bots, encom- passing both heuristic-based approaches and the state-of-the- art LLM-based agent PokéLLMon (Hu et al., 2024b). To gauge PokéChamp's 's versatility, we conduct performance analyses in two popular game modes: Generation 8 Ran- dom Battles and Generation 9 OverUsed Meta. These for- mats present distinct challenges, with Random Battles test- ing adaptability to unpredictable team compositions and OverUsed (OU) format battles examining strategic depth with carefully crafted teams. Finally, to assess real-world applicability, we pit PokéChamp against human players in online ladder battles, providing insights into its perfor- mance against skilled opponents a dynamic, competitive environment.
We argue that an informative prior can help constrain mini- max search to the space of human strategies. LLMs, trained on diverse datasets that include Pokémon-related informa- tion, offer a promising foundation for this approach. To harness the potential of LLMs effectively, we seek to de- velop an agent capable of:
Our results demonstrate that PokéChamp significantly out- performs existing bots and Al agents, achieving a 76% win- rate against the strongest LLM--based bot and an 84% win- rate against the most advanced heuristic bot in the Gener- ation 9 OverUsed Meta. Notably, PokéChamp using the open-source billion parameter Llama 3.1 model consis- tently wins (64%) against the prior strongest LLM-based bot utilizing GPT-4o, highlighting the effectiveness of our approach even with smaller language models. In online lad- der battles, P PokéCham attains an expert-level projected Elo rating of 1300-1500, placing it within the top 30%- 10% of competitive players. This performance demonstrates the agent's 's ability to compete at a high level against skilled human opponents in a dynamic. partially observable envi- ronment.
1. Proposing strategic actions to provide diverse and human-like strategies.
2. Accurately modeling the opponent based their move history, team composition, and skill level.
3. Evaluating and reflecting internally on planned game trajectories.
To achieve these objectives, we introduce PokéChamp competitive language agent that achieves comann-expert level performance in two-player turn-based Pokémon battles. PokéChamp leverages a large language model to power minimax tree search algorithm by replacing three of its key modules with LLMs: (1) player action sampling, (2) op- ponent modeling, and (3) value function estimation. This enables our agent effectively utilize gameplay history and human knowledge to reduce the search space and tackle partial observability. Within our framework, the LLM func- tions as black box, allowing for flexibility in selecting any frontier model based on budget and computational re- sources. Our approach does not require additional training or fine-tuning of LLM on Pokémon-specift data, relying instead on the LLM's s pre-existing knowledge and our novel integration with game-the planning algorithms.
In addition to our main contributions, we present several supplementary advancements that enhance the scope and impact of our research. We introduce the largest Poké- mon battling dataset to date, encompassing over 3 million games, with more than 500,000 high Elo matches, provid- ing an unprecedented resource for analyzing competitive play patterns and strategies. To rigorously evaluate bat- tling proficiency, we develop a comprehensive series of benchmarks derived from real player data and meticulously crafted puzzles, designed to assess specific battling abilities and decision- -aainn skills. Furthermore, we implement crucial updates to the local game engine, significantly im- proving its accuracy and performance, thereby ensuring a more faithful representation of official battle mechanics and enabling more reliable simulations and evaluations.
To enable effective planning, we developed a world model that approximates game transition and addresses the chal- lenges of partial observability and intricate game mechanics. This world model incorporates a one-step lookahead that mathematically computes core game dynamics and lever- ages historical data from real player games to estimate likely stats for the opponent's 's team. Our Pokémon battling dataset, containing over 3 million games across various skill levels and game modes, provides rich source of information for opponent modeling and strategy development.
Through PokéChamp we demonstrate the potential of in- tegrating LLMs with game-theoretic planning algorithms to achieve expert-level performance in complex, partially observable environments without task-specific training. Our work opens new avenues for research in competitive multi- agent settings and pushes the boundaries of Al performance in strategic game play.
We evaluate PokéChamp through a comprehensive set of experiments designed to assess its performance across vari-
2

---

PokéChamp: an Expert-level Minimax Language Agent
2. Competitive Pokémon
type once per battle, significantly altering type matchups and strategic considerations. Furthermore, the sheer num- ber of possible Tera types (all 18 Pokémon types) for each Pokémon drastically increases the state and action spaces, As a result, agents need to navigate an evolving landscape of popular Pokémon and counter-strateiies, balancing indi- vidual strengths with synergistic team compositions while accounting for the dynamic type changes introduced by Terastallization.
Competitive Pokémon battling presents a complex, par- tially observable Markov game environment with a vast state space. Two players strategically deploy teams of six Pokémon in tum-based combat, with only one Pokémon active per player at a time. Each turn, a player can choose to attack, reducing the opponent's ' Pokémon's 's health points, or switch to another Pokémon in their team. The game concludes when all Pokémon on one side have their health points reduced to zero. The state space is immense, with an estimated 10354 possibilities for the first turn alone, stem- ming from over 1000 Pokémon species with varied types, stats, and movesets. Players face asymmetric observation spaces, having complete information about their own team but only partial information about their opponent's. Strate- gic decisions must therefore be made under uncertainty.
Each player has total clock time of 150 seconds for the entire match. In addition, each turn has an incremental time limit of 15 seconds. If a player exceeds either the total clock time or the incremental turn time, they automatically lose the match. This time constraint adds another layer of com- plexity, requiring efficient recision-makinn and preventing exhaustive search strategies.
Team construction, or "teambuilding" (as illustrated in Fig- ure 3), is critical pre-battle strategic element. For each Pokémon, players configure moves, abilities, held items, nature, and a specific distribution of Effort Values (EVs) and Individual Values (IVs) that determine the Pokémon's 's statis- tics. The observation and action space are depicted in Figure 2. The action space consists of move selection, Pokémon mechanies like Terastal- lization. The transition function incorporates stochastic elements like move accuracy and damage calculation, while rewards are binary (win or loss). The competitive meta- game evolves as players discover new strategies, making adaptation essential. This complexity, combined with partial observability, makes competitive Pokémon an ideal testbed for game-theoretic research.
3. Mathematical Formalization
We can formalize Pokémon battles as a partially ob- servable Markov game (POMG) defined by the tuple (S, VY A, B, H, P.r), where is the latent state space. X' and y are the observation spaces (infosets) for the max- player and the min-player, respectively. The observation contains only information accessible to each player. A and B are the action spaces for the max pla-er and min-player. H is the horizon length. P S x A x B- - A(S) is the transition function, where P(s'|s, a, 6) denotes the probabil- ity of transitioning to state s' if starting from state s, and the max-player and the min-player take action a, b respectively. : S x A x B + [0. 1] is the reward function. We say POMG has a free structure and perfect recall if the follow- ings hold. (1) Tree structure: for any state Sy at step h, there exists a unique history (#1, 41, bi..., 8h 1h-1, -1) of past states and actions that leads to 5h. (2) Perfect recall: any infoset va of the max-player at step h. there exists a unique history (11, a1 b1, .i.. -h-1, 1, Qh-1, 0 of past states and actions that leads to z4. The same condition applies sym- metrically to the min-player. Throughout this paper, we assume the learner be the max-player who maximize the reward, and let the opponent be the min-player.
switching, and generation-spect
2.1. Pokémon Showdown Platform
Pokémon Showdown is a widely-used online battle simula- tor that implements the official battle mechanics, providing a platform for various competitive formats. It allows players to engage in battles without the need for in-game breeding or training, facilitating rapid experimentation and compet- itive play. Showdown supports multiple battle formats, in- cluding Singles, Doubles, and various tiered formats that group Pokémon based on their usage and perceived strength. The platform's 's accessibility and faithful recreation of game mechanics have made it a cornerstone of the competitive Pokémon community and valuable tool for AI research in complex game environments.
In Pokemon battle, a state contains the history of the cur- rent game and the complete information of both players' Pokémon teams (including Pokemon health, stats, items, status ailments, and other attributes), while an observation contains the history of the current game and the complete information of agent's 's team but only the partial information of the opponent's 's team. That is, an observation can corre- spond to multiple underlying states. The Pokemon battle is tree-structured POMG with perfect recall.
Generation 9 OverUsed (OU), the latest generation's 's most popular competitive format, exemplifies the intricate balance between diversity and strategic depth in Pokémon battling. The OU format bans exceptionally powerful Pokémon while maintaining a wide pool of viable options, creating a rich and evolving metagame. This format introduces mechanics like Terastallization, which allows Pokémon to change its
Policy and Nash equilibrium A policy for the max- plx-player is denoted by p : X + A(A), which is a map from ob-
3

---

PokéChamp: an Expert-level Minimax Language Agent
Dlane
LLM
Drenkeak
LLM Action Sampling: Move: Dark Puse
Opporent:
Move: Flamethrower
Other 1281
Hatorical Statiaics
Obsrvvtion
Historical Tares
Approsimate State Transition Hauritiic
Game Engin to Text
One Stop Lookahoad Promps
Observation
Historical Taris Prompt
Figure 2. PokéChamp uses one-step lookahead prompts to gain admissible heuristic information regarding the likely effect of actions under the current metagame.
Curcer Gane State
Plaar Acsion Sampll
Levfl
Dirane
He-Sen Sower
60
Figure 3. An example of teambuilding for competitive Pokémon. A player must decide six Pokémon for their team. For each Pokémon, a player must configure the item, ability, moves, stats (EVs/IVs), and nature.
Report K'imes
Genented
servation space to the space of distribution over actions. Similarly, a policy for the min-player can be denoted as v: v:y V A(B). Denote the game trajectory over Hl steps to be (51, (s1,a1,0) a1, bi. bi..... 8 0z by). ) then the value function for policy pair (n, is defined as:
Figure 4. Poké ChkéChamp replaces three components of minimax tree search with LLM-based generations: (I) sampling potential actions for the player corresponding to the first part of the edge between states., (2) modeling the opponent and sampling opponent actions corresponding to the second part of the edge hetween states, and (3) generating a potential game state value based on the depth K cutoff. PokéChamp provides the action with the best minimax value to be used battle.
VKP
Epp,
Σ r(s an bb)
of Pokémon battles, including partial observability, turn- based actions, and the goal of maximizing expected rewards, while providing the necessary context for the methodology section.
where the expectation is taken over the trajectories following policies v.
The Nash equilibrium defined as a pair of policies ()* that no player has incentive to change her strategy while the other players keep theirs unchanged. That is,
4. Agent Architecture and Application
inf VN " (s) Vaa .M" (s) sup VAN" (s)
PokéChamp implements novel approach that leverage the power of large language models and integrate them into minimax tree search. We first describe the generic framework which applies to general two-player zero-sum games. We then discuss specializations we make to adapt our framework for Pokémon battle.
The minimax theorem holds in our setting, and Nash equi- librium achieves the minimax value:
sup inf V (s) = Va .²" (s) = inf sup (s) (1)
Combining two equations, equilibrium is also the optimal policy against the adversarial opponent. This formalization captures the essential elements
not hard to see that Nash
Minimax tree search framework. Due to computation constraints and tight inference time requirement, we focus on pure strategies and compute the best action that is safe

---

PokéChamp: an Expert-level Minimax Language Agent
even against the adversarial play of the opponent. The overall strategy corresponds to minimax tree search which is described at Figure 4:
than the latent state S/. To infer hidden information and ap- proximate latent state, we leverage the statistical data from Pokémon Showdown, including Pokémon move pools, EV spreads, item usage, etc., aggregated from gameplay over a given period. Additionally, we incorporate L.LM predic- tions to estimate hidden opponent variables, particularly the attack (A) and defense (D) stats, inferred from game history. The relationship between these stats and damage output captured by the following equation:
At step h, the learner observes an infoset Th. chooses an action (z. and the opponent chooses an action b The game transition to the new infoset Th+11 We repeat this process until game terminates at step H, which leads to a tree of depth H h where each path of the tree corresponds to 3( H h) -tuple (an: bb. #h+11-...... aH-1. - b, In the end, we receive the reward r(wn) for every infoset at the step H. The minimax tree search algorithm recommend
(3) 50 ((3 Level +2) Power. A 1
))+2) -M
Damage
+2)
action ūz to the learner at Th according to the following criteria:
where A represents the attacker's relevant attack stat and D the defender's 's relevant defense stat. The M term en- capsulates various game-specific modifiers (detailed in the appendix, equation 2). After predicting 8h, we obtain $h+1 by simulating the game using our local Showdown simula- tor. Notably, the actual Pokemon game features stochastic transition-for example, many attack moves have a prob- ability of missing. To alleviate computational burden, we adopt a simplified approach that computes the expected value within these transitions.
0j = arg max min Rzzap1 an
max min (((H) aN-1 b
On
is not hard to see that, the size of the tree grows exponen- tially with respect the search depth H h, and thus comprehensive search over the entire tree quickly becomes infeasible.
We propose a novel minimax agent framework that inte- grates L.MS into three key modules of classical minimax tree search, significantly enhancing its performance:
Player action sampling For a given infoset Ih at step he PokéChamp generates a set of candidate actions {at ] C A to form the edges of the minimax search tree. The input prompt for action sampling includes:
Player Action Sampling: We provide the LLM with key information about the current infoset and prompt it to sample a small set of viable actions for tree expan- sion. This aggressively prunes the search tree, reducing computational costs.
Team strategy: An LLM-generated overall strategy based on both players' teams.
Opponent Modeling:
Similarly, we prompt the LLM to sample the most likely opponent actions. Unlike player action sampling, opponent modeling is more complex due to partial observability, requiring the LLM to infer hidden states based on the player's observations.
Observable state: Iy E X. including the player's team, items, and visible opponent Pokémon.
Battle history: Information from the last N turns, representing the perfect recall assumption.
Value Function Estimation: Instead of expanding the search tree to the end of the game, we limit expansion to a depth of le steps and use L.MS as value function to evaluate infosets at this time step, replacing termi- nal rewards. It is challenging to estimate this value accurately through traditional methods, as it requires deep understanding of game mechanics and effective state embeddings. In contrast, LLMs leverage common sense and gameplay knowledge from the internet, serv- ing as an effective approximation for value functions.
Approximate State Transition Heuristic: We use the approximate state transition to create an admis- sible heuristic for each player action. Using a one step lookahead from the approximate state transition, we can calculate the least number of turns to knock out (KO, term for defeating the opponent's current Pokémon). This information is commonly available and used by players on the online ladder on Pokémon Showdown.
Available actions: The set of legal actions An (2x) C A under the current info set.
4.1. PokéChamp
In addition LLM generated actions, also include few candidate actions from our tools, including (1) the top move choice from our one step lookahead and (2) the top switch choice from the Abyssal bot.
We now describe the implementation of PokéChamp and the design of its three key modules.
Approximate game transition One challenge in our system is simulating game transitions, represented as Ph an bR), under partial observability. In our set- ting, the learner only has access to the observation T4 rather
Opponent modeling To address the partial observability of the opponent's actions and hidden state information, we employ a combination of historical data analysis and LLM- based prediction:
5

---

PokéChamp: an Expert-level Minimax Language Agent
Average Game Length by Elo
20.00
gentnu
gentrandombatttle
25.88
+25.66
33.29
.24.67
21.48
3000
20.7
119.56
1000-1199
1200-1399
1400-1509
1600-1759
1800+
Bracket
Figure 5. Left: Elo distribution for collected battles across game formats. Right: Relationship between game length and player Elo rating.
Stat estimation: For unknown opponent stats (A and D in equation 4. 4.1), we use historical data to estimate the likelihood of different stat distributions. This al- lows us approximate the true state s/ S given the observation y/. E y.
Ps (s an, bn) and opponent policies vn (-1g)) The dataset includes over 3 million game replays across various formats, with more than 500,000 high-Elo games (Elo 1600). It contains detailed information team composi- tions, move choices, and battle outcomes, providing robust foundation for our analysis and modeling efforts.
Action prediction: The LLM generates likely oppo- nent actions by E B based on a prompt similar to the action sampling process, but focused the opponent's perspective.
Figure 5 illustrates the distribution of Elo ratings and game lengths across different formats in our dataset. The left panel shows stacked histogram of Elo ratings for various game formats, revealing multi-modal distribution with two primary modes approximately 1 1150 and 1350 Elo. This distribution provides insights into the skill levels of play- ers across different game modes. The right panel presents a scatter plot of average game length versus Elo rating, demon- strating a general trend where higher Elo matches typically have longer durations. This relationship suggests that more skilled players engage more complex and protracted bat- tles.
Value function Due to the computational constraints of live gameplay (150 seconds per player, with up to 15 ad- ditional seconds per turn), we employ an LLM-generated value function to evaluate leaf nodes at depth & of our mini- max tree. The LLM generates a score based on the following criteria:
Effectiveness current moves, num- ber of remaining Pokémon, and win probability.
Positive factors:
Negative factors: Excessive switching, opponent move effectiveness, speed disadvantage, opponent's s remaining Pokémon, and strength of remaining oppo- nent Pokémon.
5.1. Action Prediction
To evaluate the effectiveness of our modules of player ac- tion sampling and opponent modeling, conducted ex- periments on predicting human actions using our collected dataset. This task is particularly challenging due to the partial observability of the game state.
We then compute the minimax optimal action On according to:
ār = arg max min Exs41 4% by.
max min Emra V (In++)
4k butt-1
The replay data is collected from a spectator's perspective, where key information such as EV spreads and item choices for both players remains hidden. This contrasts with the standard player perspective, where only the opponent's 's team information is obscured, while the player's own team de- tails are fully accessible. To bridge this gap, we reverse- engineered the moves, team compositions, and stats from the replay data. We then supplemented this information with additional switching and move options based on historical likelihood. We use this processed data as proxy to real game- play with standard player perspective. This processed data was then fed into our prompt generator, enabling our bot to predict both player and opponent actions.
By combining these three components action sampling. opponent modeling, and value function approximation PokéChamp effectively navigates the complex, partially observable state space of Pokémon battles, approximating optimal play within the constraints of real-time gameplay.
5. Evaluation
In this section, we describe our evaluation methods which evaluate PokéChamp via both offline dataset and online games on Showdown platform.
Pokémon Battling Dataset. We compiled comprehen- sive dataset of over 3 million Pokémon battles from the Pokémon Showdown platform. This dataset serves as a rich source of information for estimating transition probabilities
We compared the accuracy of these predictions with the historical data across various Elo ratings. Table presents the results of our action prediction experiments. The player
6

---

PokéChamp: an Expert-level Minimax Language Agent
Table Player and opponent action prediction accuracy by Elo rating. Random baseline performance is 7% for player prediction less than I for opponent prediction due to partial observability.
Elo
Top
Top
Top 3
Top 4
Top 5
1200
30%
40%
48%
53%
58%
Player's
1400
26%
23%
30%
32%
43%
action
1600
27%
30%
39%
44%
53%
prediction
1800
30%
42%
55%
62%
66%
1200
16%
30%
40%
46%
53%
Opponent's
1400
16%
17%
20%
26%
39%
action
1600
13%
21%
26%
35%
40%
prediction
1800
15%
29%
40%
50%
53%
iebrata Hardis
1:50
0.41
1:50
1.24
6.40
Figure Left: PokéChamp utilizes Terastallization to change type matchups. Right: PokéChamp employs Dynamax to in- crease hit points and move power, enabling consecutive knockouts
Figure 7. Left: Pairwise win rates in Gen 9 OU. Right: Impact team composition on win rates.
prediction accuracy for PokéChamp varies between 26% and 30% as Elo increases, while the opponent prediction accuracy is lower, ranging from 13% to 16%. These re- sults demonstrate that predicting opponent actions more challenging given the limited state information available.
guarantee a 100% win rate in every instance.
The results of this benchmark show PokéChamp's perfor- mance of 86% winrate is higher than PokéLLMon's of 76%. Our method achieves 10% higher win rate. Since this is a constrained setting that does not require Pokémon switching, key factor contributing to our agent's 's superior performance is its effective utilization of one-step lookahead.
We can see that LLM prediction is significantly better than random prediction (7% for player, 1% for opponent) in both cases. It's worth noting that there may be multiple "cor- rect" actions in many situations, as several strategies could be equally viable. Therefore we included in the table the performance for top k prediction, and observe the accuracy increases significantly as k increases.
Puzzles for special mechanics: Terastallization and Dynamax. To evaluate PokéChamp's 's ability to utilize generation-specife game mechanics, we developed puzzles focusing on Terastallization and Dynamax. These mechan- ics introduce additional complexity to the state space S and action space A. Our world model and prompting mecha- nism for PokéChamp allow to understand and use these generation-specife game mechanics effectively. The agent is informed about the mechanics' effects, and the one step lookahead provides information about different outcomes when using these mechanics.
5.2. Evaluating on Small Game Puzzles
To assess PokéChamp's 's ability navigate complex game states and make optimal decisions, we developed a series of game puzzles as benchmarks. These puzzles are designed to test the agent's 's understanding of core game mechanics and its ability to approximate optimal policies μ² and v²"
1y1 Battles. We created a benchmark of 1,000 1v1 battle scenarios to assess PokéChamp's 's ability to find optimal move sequences. Each scenario 81 E S₁ is carefully se- lected ensure feasible win condition, allowing us to evaluate the agent's performance in finding the optimal pol- icy 1²" We selected matchups from the genSrandombattles meta, with each matchup consisting of one Pokémon on each team. To ensure a feasible win condition, we rejected samples that could not be won by the Abyssal bot. However, due to the stochastic nature of move damage, this does not
Figure 6 illustrates P Pok PokéChamp's proficiency in leverag- ing these mechanics to gain strategic advantage. In the left panel, PokéChamp demonstrates its understanding of the changed weakness of Roaring Moon after Terastalliza- tion, deciding to switch after Glimmora is chosen. In the right panel, PokéCham employs Dynamax to increase its hit points and move power, enabling it to knock out two Pokémon in succession.
These benchmarks demonstrate PokéChamp's ability to

---

PokéChamp: an Expert-level Minimax Language Agent
Table 2. Performance in Gen 9 battles with terastallize meehanic and custom teams.
Win Rate vs.
Method
LLM
Elo
Avg. Turns
Abyssal (%)
PokéChamp
GPT-4o
84
1268
15.7
PokéChamp
Llama 3.1:8b
56
1204
16.9
PokéLLMon
GPT-4o
40
1020
22.6
Abyssal
N/A
N/A
1117
17.9
One Step Lookahead
N/A
44
1107
17.9
Max Power
N/A
16
885
19.5
Random
N/A
0
399
21.2
Table Gen OU mirror matchups without tcrastallize mechanic.
Random: A bot that selects actions randomly.
Win Rate
Method
LLM
PokéChamp with GPT-4o achieves the highest perfor- mance, with an 84% win rate against the Abyssal bot and an Elo rating of 1268. Notably, PokéChamp using Llama 3.1:8b outperforms PokéLLLMon powered by GPT- 4o, demonstrating the effectiveness of our approach even with smaller language models. Figure 7 illustrates the pair- wise win rates between methods (left) and the impact of team composition on performance (right). To isolate the effects of team composition and the terastallize mechanic, we conducted additional experiments using mirror matchups without terastallize. Table 3 presents these results. These results suggest that team composition significantly impacts performance, with PokéChamp achieving 90% win rate against Abyssal in mirror matchups.
Abyssal (%)
PokéChamp
GPT-40
90
PokéChamp
Llama 3.1:8b
83
PokéLLMon
GPT-4o
60
One Step Lookahead
N/A
56
navigate complex game states and make strategic decisions that approximate optimal policies in the presence of special mechanics. The agent's 's performance in these puzzles high- lights its capacity to adapt to various game scenarios and utilize advanced mechanics to gain a competitive edge.
5.3. Evaluating on Full Games
In this section, we evaluate PokéChamp's performance in the Gen OU (OverUsed) format, which includes the terastallize mechanic and uses custom teams. We evalu- the
Against human players on the online ladder. To assess rorl-world performance, we evaluated PokéChamp on the Pokémon Showdown online ladder between September Ist and September 7th, 2024, where faced human opponents in the Gen 9 OU format. Each player has minutes and 30 seconds to play the game and the player will lose im- mediately if one fails to make decision within the time limit (similar to chess clock). Players can receive up to 15 additional seconds added to their time per move. For about one third of the games, PokéChamp lost by exceed- ing the turn time limit. Within the remaining two thirds of games, PokéChamp achieved a 76% win rate. After 50 games, PokéChams reaches Elo score above 1300 the online Showdown ladder. If removing losses due to timeout, we can estimate its Elo rating, where PokéChamp will achieve an Elo rating of 1300-1500, which places it in the top 30%- 10% of players. We then use the win rates of each bot (PokéLLMon, Abyssal, etc.) against PokéChamp to estimate the Elo range of each method Figure 1.
ate PokéChamp against other baselines well online ladder against real human players. Additional experi- ments for the Gen 8 Random Battles format are provided in Appendix D.
Against other bots and agents. We compare PokéChamp against -fate-of-the-art open-source bots and LLM-based agents. Each experiment consists of at least 25 matches between any two methods, resulting in minimum of 100 games per method for Elo calculations. The LLM agents utilize either Llama3.1:88 (Dubey et al., 2024) or GPT-4--2024-05-13 (Achiam al., 2023). Our baselines include:
PokéLLMon (Hu et al., 2024b): An LLM-based agent using self-consistency prompting.
Abyssal Bot: rule-based heuristic bot used in official
Pokémon games.
On March 3rd, 2025, investigated optiiiop vari- ant of our method, PokéChamp Fastt, where the LLM dynamically chose between using the calculated one-step lookahead action or performing a truncated minimax search based on the current game state. This was designed to al- leviate time constraint issues. However, the resulting Elo
One Step Lookahead Bot: Our admissible heuristic bot using the approximate state transition from Sec- tion 4.1.
Max Power Bot: A bot that selects moves based solely on power level.
8

---

PokéChamp: an Expert-level Minimax Language Agent
performance ranged between 1 1150-1310 without conver- gence, decrease compared to our standard method. We attribute this performance difference to covariate shift in the competitive metagame over time. Specifically, the LLM's pre-training data, which has a fixed cutoff date, likely biases its decision- makion even when provided with up- dated historical statistics about the current metagame. This suggests that the LLM's inherent prior knowledge can out- weigh real-time statistical information, particularly when deciding between strategic exploration (minimax search) and exploitation (one-step lookahead).
world settings include Minecraft (Wang et al., 2023) and Spring (Wu et al., 2024); and finetuning on human tran- scripts enables Diplomacy play (, FAIR). Complementary approaches study distillation (Nalty & Rosenthal, 2024), online self-play finetuning (Zhou et al., 2024), and reward shaping with LLM feedback (Klissarov et al., 2023).
Prompting and Planning. Prompt strategies-chain-o0 oh thought (Wei et al., 2022), self-connistency (Wang et al., 2022), Tree-of-Thoughts (Yao et al., 2024), and ReAct (Yao et al., 2022) reasoning, while search and RI. fur- ther enhance control. RAP treats LLM as both plan- ner and world model (Hao et al., 2023); TS-LLM applies AlphaZZero-tylle tree search during decoding (Feng et al., 2023); and commonsense planners query LLMs as knowl- edge sources (Zhao et al., 2023). Self-correction methods such as SCoRe (Kumar et al., 2024) and Reflexion (Shinn et al., 2024) refine policies via linguistic feedback without heavy finetuning.
We observed that PokéChamp struggled against two spe- cific strategies: stall tactics and excessive switching. These challenges likely stem from the limited lookahead depth necessitated by time constraints and the difficulty of accu- rately modeling such strategies in-context. Further analysis of these weaknesses is provided in Appendix C.I and C.2. During live demos, we note that games played against hu- mans who knew they were playing against PokéChamp were able to determine the excessive switching limitation and tailor an adversarial strategy to take advantage of these limitations. Thus, we emphasize anonymity for accurate performance analysis.
7. Conclusion
In this paper, we introduce PokéChamp, which augments minimax tree search with the following L.LM-based compo- nents: (1) action sampling, (2) opponent modeling, and (3) state value function. PokéChamp achieves state-of-the art performance against heuristic and LL-M-based bots and expert performance against real players on the online ladder. The objective of our framework lies at the intersection of imitation learning, best response estimation, and Nash equi- librium approximation. While imitation learning estimates the best response to the meta-game, and computing the ex- act Nash policy may be intractable, PokéChamp aims to strike balance between these approaches. Our max-min formulation in the minimax search finds conservative ac- tion that approximates the true best response, although the exact relationship between our method and the optimal best response remains an open question for future investigation. Further performance enhancements are currently limited by the accuracy of opponent modeling and the method's s online computational budget. By increasing the breadth and depth size of the search, we expect to see further improve- ment's 's to the performance of the method. Additionally, our work can be taken advantage of adversarially due to static opponent modeling, Our work leaves open challenges in opponent modeling and generative minimax planning for future work exploring competitive multi-agent settings and future superhuman performance in Pokémon battling. Our benchmark explores LLMS and test time planning for com- petitive POMG in Pokémon battling. However, we provide a generalized framework of action sampling, one step looka- head world modeling, opponent modeling, and value that may be easily applied to other frameworks.
In summary, our evaluation demonstrates that PokéChamp achieves state-of-the ofte-off-the-ar performance in Pokémon battles, outperforming both heuristic and LL.M-based baselines while exhibiting expert-level play against human opponents. These results underscore the effectiveness of our approach in leveraging LLMs for complex. partially observable game environments.
6. Related Work
Competitive Games. RL self-play has produced superhu- man agents in Chess (Campbell et al., 2002; Silver et al., 2017), Go (Silver et al., 2016), Poker (Brown & Sandholm, 2019; 2018), StarCraft II (Vinyals al., 2019), and Dota 2 (Berner et al., 2019), and has recently scaled to domains such as Street Fighter (Li et al., 2024) and Gran Turismo Sport (Wurman et al., 2022). For Pokémon, prior work combines self-play with heuristics to reach competitive strength (Huang& & Lee, 2019; Whidden, 2023). In con- trast, PokéChamp attains expert-level play without any explicit training.
Language Agents for Games. LLMs remain weak plan- ners-for example, they still miss the Nash strategy in Tic- Tac-Toe (Topsakal & Harper, 2024) but a growing liter- ature explores augmentations (Hu et al., 2024a). Prompt- based agents have been demonstrated for Pokémon (Hu et al., 2024b), StarCraft (Ma et al., 2023), and Avalon (Shi et al., 2023; Stepputtis et al., 2023); tool-use extensions tackle NetHack (Küttler et al., 2020; Jeurissen et al., 2024); open-
9

---

PokéChamp: an Expert-level Minimax Language Agent
Acknowledgement
(FAIR),, M. F. A. R. D. T., Bakhtin, A., Brown, N., Di- nan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., Jacob, A. P., Komeili, M., Konath, K., Kwon, M., Lerer, A., Lewis, M., Miller, A. H., Mitts, S., Renduchintala, A., Roller, S., Rowe, D., Shi, W. Spisak, J., Wei, A., Wu, D., Zhang, H., and Zijlstra, M. Human-level play the game diplomacy by combining language models with strategic reasoning. Science, (0966624):1067--11 1074, 2022. doi: 10.1126/ science.ade9097, URL https: pettps:/wwer //www../wwww./ http:://wwwwscceeneeee.. science. org doi /ab///b/////// 0. 1126/ het///cienneeee ade aete///ceeeneeee....... 6
The authors acknowledge the support of Office of Naval Research Grant N00014-22-1-2253, National Science Foun- dation Grant NSF-OAC-2411299, the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE-2039656, and computational resources from Princeton Language and Intelligence (PLI).
Impact Statement
In our work, we use a small number of games with our method on the community-maintainee open-source com- petitive Pokémon platform. Pokémon Showdown. As the popularity of AI agents for Pokémon increases, we urge researchers to limit use of the online ladder evaluation to avoid data contamination and overwhelming human users of the platform with AI opponents. Rather, we recommend that researchers compete Al agents against other Al agents.
Feng, X., Wan, Z., Wen, M., MeAleer, S. M.. Wen, Y., Zhang, W., and Wang, J. Alphazero-like like tree-search can guide large language model decoding and training. arXiv preprint 1t.Xiv:2399.717777. 2023. 6
Foerster, J. N., Chen, R. Y., Al-Shedivat, M., Whiteson, S., Abbeel, P., and Mordatch, I. Learning with opponent- learning awareness. arXiv preprint (tt:::709.0333. 2017.
References
Hao, S., Gu, Y., Ma, H., Hong, J., Wang. Z., Wang, D. Z., and Hu, Z. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305. 14992, 2023. 6
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L.,, Almeida, D., Altenschmidt, J., Altman, S.,
Anadkat, S., al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 5.3
Hu, S., Huang, T., Ilhan, F. Tekin, S., Liu, G., Kompella, R., and Liu, L. A survey on large language model-based game agents. arXiv preprint arXiv:2404.02033 2024a. 6
Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D.,, Fischer, Q., Hashme, S., Hesse, C., et al. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv: 1912.0668//0 2019. 1, 6
Hu, S., Huang, T., and Liu, L. Pok) 'ellmon: A human- parity agent for pok) 'emon battles with large language models. arXiv preprint 2rXiv:2402.01/18, 2024b. 1, 5.3, 6, B.4, 4
Brown, N. and Sandholm, T. Superhuman ai for heads- up no-limit poker: Libratus beats top professional.. Science, 359(6374):418-4))
2018. doi: 10.1
science.aaol733. URL hitps: http:::/wwwwwwiinn
Huang, D. and Lee, S. A self-play policy optimization approach to battling pokémon. In 2019 IEEE conference on games (CoG), pp. 1-4. IEEE, 2019. 6
org/doi/abs /0/0100.0000 http:///bbb/o1.1///cccieeeee aao1733. 1,
Brown, N. and Sandholm, T. Superhuman ai for multiplayer
Jeurissen, D., Perez-Liebana, D., Gow, J., Cakmak, D., and Kwan, J. Playing nethack with 1lms: Poten- tial & limitations as zero-shot agents. arXiv preprint arXiv:2403.0006 2024.
poker. Science, 365(6456)8885889, 2019, doi: 10.1
/www sciencee.ee.....eeeeeee.e / http:///.abba.......///ceeeeeeee aay2400 1,
science.aay2400. URL httpp://www.w.tt.... /00
Klissarov. M., D'Oro, P., Sodhani, S., Raileanu, R., Bacon, P.-L., Vincent, P., Zhang, A., and Henaff, M. Motif: Intrinsic motivation from artificial intelligence feedback. arXiv preprint arXiv:2300.000 2023.
Campbell, M., Hoane Jr, A. J., and Hsu, F.-h. Deep blue. Artificial intelligence, 134(1-2):57-83, 2002. 6
Ding, Z., Su, D., Liu, Q., and Jin, C. A deep reinforcement learning approach for finding non-exploitable strategies in two-player atari games. arXiv preprint arXiv:2207.0889, 2022.
Kozuno, T., Ménard, P., Munos, R., and Valko, M. Learn- ing in two-player zero-sum partially observable markov games with perfect recall. Advances in Neural Informa- tion Processing Systems, 34:1 14:11987--111 -1998, 2021.
Dubey, A.. Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The Ilama herd of models. arXiv preprint arXiv:2407.2178 2024. 5.3
Kumar, A., Zhuang, V., Agarwal, R., Su, Y., Co-Reyes, J. D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., et al. Training language models to self-correct via
10

---

PokéChamp: an Expert-level Minimax Language Agent
reinforcement learning. arXiv preprint arXiv:2409. 2024. 6
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575 (7782)3350-354 2019. 1,
Küttler, H., Nardelli, N., Miller, A., Raileanu, R., Selvatici, M.. Grefenstette, E., and Rocktäschel, T. The nethack learning environment. Advances in Neural Information Processing Systems, 33:7671-766 2020. 1,6
Wang. G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A. Voyager: An open- ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 2023. 6
Li, W., Ding, Z., Karten, S., and Jin, C. Fightladder: A benchmark for competitive multi-agent reinforcement learning. In Forty-firsi International Conference on Ma- chine Learning, 2024, http:: openreview....e net. foru/form?2 1==000446601. 6
Wang, X., Wei, J., Schuurmans, D., Le, Q. Chi, E.,, Narang, S., Chowdhery, A., and Zhou, D. Self-consistency im- proves chain of thought reasoning language models. arXiv preprint arXiv:2203.11117, 2022.
Ma, W., Mi, Q., Yan, X., Wu, Y., Lin, R., Zhang, H., and Wang. Large language models play starcraft ii: Bench- marks and chain of summarization approach. arXiv preprint arXiv:2312.11865 2023. 6
Wei, J., Wang. X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et Chain-of- ofii--of-hhugghh prompting elicits reasoning in large language models. Advances in neural information processing systems, 01-24824-24831
Nalty, C. and Rosenthal, S. mon
Pokéll-
trainer: Llm model distillation. htwpaaaww.giiiuubiooo////ooeeeee
2022.
2024. 6
Whidden, P. Training ai play pokemon with reinforce- ment learning, 2023. URL Jiu61TI. 6
Shi, Z., Fang, M., Zheng, S., Deng, S., Chen, L., and Du, Y. Cooperation on the fly: Exploring language agents for ad hoc tcamwork the avalon game. arXiv preprint arXiv:2312.175155 2023. 6
be/
9bUk1 9bKK
Wu, Y., Min, S. Y., Prabhumoye, S., Bisk, Y., Salakhutdinov, R., Azaria, A., Mitchell, M., and Li, Y. Spring: Studying papers and reasoning to play games. Advances in Neural Information Processing Systems, 36, 2024. 6
Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. Reflexion: Language agents with verbal rein- forcement learning. Advances Neural Information Processing Systems, 36, 2024.
Wurman, P. R., Barrett, S., Kawamoto, K., MacGlashan, J., Subramanian, K., Walsh, T. J., Capobianco, R., Devlic, A., Eckert, F., Fuchs, F,, et al. Outracing champion gran turismo drivers with deep reinforcement learning. Nature, 602(7896):223-228) 2022.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. Mastering the game of go with deep neural networks and tree search.
nalure, (29(758))44444499 2016. 1,
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022. 6
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre,, L., Kumaran, D.. Grae- pel, T., al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm, arXiv preprint arXiv:1712.018855 2017. 1,
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. Cao, Y... and Narasimhan, Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024. 6
Stepputtis, S., Campbell, J., Xie, Y., Qi, Z., Zhang, W. S., Wang, R., Rangreji, S., Lewis, M., and Sycara, K. Long- horizon dialogue understanding for role identification in the game of avalon with large language models. arXiv preprint 1.Xi:22311.05720 2023. 6
Zhao, 7., Lee, W. S., and Hsu, D. Large language models as commonsense knowledge for large-scale task planning. In Advances in Neural Information Processing Systems, volume 36, pp. 31967-31987. Curran Associates, Inc., 2023. 6
The-Third-Bing How an a.i. is becoming the world's best pokemon player, 2022. URL /tstts::/youtus
TtCCCC/RRRysiiiAAuuuuY www.:/sssssswwa-----e-- Zbileuu....
Zhou, A., Yan, K., Shhapentokh-Rottmane M., Wang, H., and Wang, Y.-.. Language agent tree search unifies reasoning acting and planning in language models. arXiv preprint arXiv:2310.04400 2023.
Topsakal, O. and Harper, J. B. Benchmarking large language model (Ilm) performance for game playing via tic-tac-toe. Electronics, 1532, 2024. 1,6

---

PokéChamp: an Expert-level Minimax Language Agent
Zhou, R., Du, S., and Li, B. Reflect-rl: Two-player online
rl fine-tuning for lms, arXiv preprint arXiv:2402//2621, 2024. 6
12

---

PokéChamp: an Expert-level Minimax Language Agent
A. Appendix
B. Background
B.1. Pokémon Showdown and Competitive Battling
Pokémon Showdown is an online battle simulator that allows players to engage competitive Pokémon battles without the need for in-game breeding or training. It implements the official battle mechanies and provides a platform for various competitive formats.
In competitive Pokémon battles, two players each bring team of up six Pokémon and engage in turn-based combat. The objective is to make all of the opponent's Pokémon faint. Each turn, players can choose to use one of their active Pokémon's moves, switch to a different Pokémon, or use an item.
Key aspects of competitive battling include:
Type effectiveness: Each Pokémon and move has one or two types, with a complex system of type matchups affecting damage calculations.
Stats:
Six core stats (HP, Attack, Defense, Special Attack, Special Defense, and Speed) determine a Pokémon's performance in battle.
Abilities: Special traits that can provide various effects during battle.
Held items: Objects that Pokémon can carry to gain additional effects or bonuses.
B.2. Battle Formats
Pokémon Showdown supports various battle formats, including:
Singles: One-on-one battles where each player has active Pokémon time.
Doubles: Battles where each player has active Pokémon simultaneously.
Random Battles: Players are assigned random teams of fully-trained Pokémon.
Tiered formats: Formats that group Pokémon based on their usage and perceived strength (e.g., OU, UU, RU).
Our research focuses primarily on the Generation OverUsed (OU) format, which is a singles format featuring the most commonly used Pokémon that are not considered too powerful for standard play.
B.3. Partial Observability in Pokémon Battles
Pokémon battles are partially observable Markov games. The partial observability stems from several factors:
Players can only see their opponent's active Pokémon, not their entire team.
The exact stats, abilitics, and movesets of the opponent's Pokémon are unknown until revealed through battle actions.
Some moves and abilities can temporarily change the game state in ways that are not fully observable to the opponent.
This partial observability adds a layer of complexity to decision decision-making and strategy, as players must make inferences about their opponent's team and potential actions based on limited information.
13

---

PokéChamp: an Expert-level Minimax Language Agent
B.4. Pokémon Battling AI
Various approaches have been developed for creating Al agents capable of playing Pokémon battles:
Rule-based systems: These include the Al used in official Pokémon games, such as the Abyssal bot, which follows pre-defined heuristies for decision-making
Search-based methods: Approaches like Expectiminimax have been attempted but face challenges due the large branching factor and enforced time limits per turn.
Machine learning approaches: Some efforts have used supervised learning on battle data to predict opponent moves and inform decision-making.
While successful in other game domains, the complexity and partial observability of Pokémon battles have made this approach challenging.
Reinforcement learning:
Recent work has explored the of large language models (LLMs) for Pokémon battling, such PokéLLMon (Hu et al., 2024b), which uses prompting techniques to generate actions based on battle state descriptions.
B.5. Evaluation Metrics
The primary metries for evaluating Pokémon battling agents include:
Win rate: The percentage of games against a specific opponent set of opponents.
Elo rating: relative skill rating system used in competitive gaming, where higher Elo indicates stronger performance against other rated players.
Average number turns: The typical length of games played by an agent, which can indicate efficiency playstyle.
These metries allow for comparison between different Al agents and human players, providing insights into the relative strengths and weaknesses of various approaches.
B.6. Damage Calculation
The damage calculator is crucial component of PokéChamp which helps use the LLM predicted information get the step lookahead, implementing key part of the transition function (s an b,). It uses the following equation to compute the expected damage for given move:
((:
Level + +2) Power ) +2)
Damage
50
Targets
PB
Weather
GlaiveRush
Critical
random (0.85,1.00)
STAB
Type
-urnn
ooherr
ZMove
TeraShield.
where:
14

---

PokéChamp: an Expert-level Minimax Language Agent
A and D are the relevant attack and defense stats
Targets is 0.75 the move hits multiple targets in doubles/triples, otherwise
Weather is 1,5 for Water moves in Rain or Fire moves in Sun, 0.5 for the opposite, 1 otherwise
Critical is 1.5 for critical hits, otherwise
STAB (Same Type Attack Bonus) is 1.5 if the move type matches the user's type, I otherwise
Type is the type effectiveness multiplier (0.25, 0.5, 1, 2, or 4)
Burn is 0,5 for physical moves if the user is burned, 1 otherwise
Other includes various move-specific and ability-based modifiers
This detailed calculation allows PokéChamp to accurately estimate the outcomes of different actions, informing its decision-making process within the minimax tree search.
An example prompt that generated by the damage calculator for all matchups provided in listing 1.
Requires switch:
dragapult vs. primarina:
dragapult outspeeds primarina
dragapult's moves:
dragondarts: 161 turns to KO opponent's pokemon
uturn: turns to ио opponent's pokemon
7 quickattack: 5 turns to ко opponent's pokemen
terablast: 5 turns to ко opponent's pokemon
dragapult's moves if opponent's primarina uses 'terastallize":
dragondarts: turns KO opponent's pokemen
n uturn: 11 turns to ко opponent's pokemon
n quickattack: 321 turns to KO opponent's pokemon
B terablast; 321 turns to KO opponent's pokemen
dragapult's moves if it uses 'terastallize' and opponent's primarina uses terastallize':
is dragondarts: 4 turns to ко opponent's pokemen
16 uturn: turns to ко epponent's pokemon
quickattack: 10 turns to ко opponent's pokemon
terablast: turns to KO opponent's pokemen
19 dragapuit's moves
uses "terastallize" and opponent's primarina does NOI use
terastallize':
30 dragondarts: 161 turns to ко opponent's pokemon uturn: turns to ко opponent's pokemon
quickattack: turns
opponent's pokemen terablast: turns to кO opponent's pokemon
24 Opponent moves: primarina
moonblast: 2 turns to Kо your pokemon
paychicnoise: turns to your pokemon
surf: turns
your pekemon
a flipturn: 10 turns to Ko your pokemon
Listing 1. Example prompt from damage calculator.
C. More Puzzles
To further evaluate PokéChamp's strategic capabilities and identify areas for improvement, we developed additional puzzle scenarios that test specific aspects of Pokémon battling.
15

---

PokéChamp: an Expert-level Minimax Language Agent
C.1. Stall Strategy
Stall strategies in Pokémon battles involve using defensive Pokémon with high HP and recovery moves to gradually wear down the opponent. These strategies often rely on status effects, entry hazards, and passive damage to win battles over many turns.
PokéChamp struggles with stall strategies due to the uncertainty they introduce in the current matchup. This uncertainty often causes Po PokéChamp to switch its current Pokémon frequently, which can be counterproductive. The agent's difficulty in handling stall strategies stems from two main factors:
Limited lookahead: The minimax tree search used by PokéChamp may not extend far enough to fully capture the long-term benefits of maintaining position against stall team.
2. Overemphasis on immediate damage: The value function Vx(*k) may not adequately account for the cumulative effects of status conditions and passive damage over many turns.
Figure (left) illustrates this behavior. In this scenario, PokéChamp initially chooses Darkrai against Blissey, recognizing that Darkrai's Focus Blast is strong against Blissey. However, the agent then decides to switch to Enamorus, which faints from entry hazards. After sending Darkrai back in, misses the first Focus Blast against Blissey. This miss increases uncertainty in PokéChamp's decision-making process, leading it switch to another Pokémon rather than maintaining its position.
C.2. Excessive Switching
Another challenging scenario for PokéChamp is when opponents employ excessive switching strategies. In this approach, opponents frequently switch their Pokémon to disrupt the agent's planning and exploit the limitations of short lookahead methods.
PokéChamp struggles to capitalize on or defend against this strategy due to several factors:
Incomplete opponent modeling: The opponent policy pn (-(-) may not accurately capture the likelihood of frequent switching.
2. Myopic decision-making The limited depth of the minimax tree search may prevent PokéChamp from recognizing the long-tern -erg-termm advantages of predicting and punishing switches.
3. Overcommitment to predicted optimal moves: Once P PokéChamp identifies a strong move against the current opponent Pokémon, it may persist in using that move even when it becomes suboptimal due to switches.
Figure 8 (right) demonstrates this weakness. In this example, P PokéChamp consistently chooses Focus Blast, a Fighting-type move, even as the opponent switches between two Pokémon that resist Fighting-type attacks. This behavior allows the opponent exploit P PokéCha okéhhamp's 's predictability and inability to adapt to the switching strategy.
Figure 8. Left: Stall strategy: PokéChamp's 's response to a stall tactic, illustrating its difficulty in maintaining consistent strategy against defensive play. Right: Opponent excessive switching strategy: PokéChamp's 's struggle to adapt to opponent's 's frequent Pokémon switches, demonstrating tendency to persist with suboptimal move choices.
These puzzles highlight areas where PokéChamp's 's decision-making process can be improved, particularly in handling
16

---

PokéChamp: an Expert-level Minimax Language Agent
long-term strategies and adapting dynamic opponent behaviors.
D. Random Battles
To evaluate PokéChamp's performance in setting with higher uncertainty and variability, we conducted experiments in the Gen Random Battles format. This format presents unique challenges as players must adapt to randomly assigned teams, testing the agent's ability to quickly assess team strengths and weaknesses.
We compared PokéChamp against various baselines in two scenarios: with and without the dynamax mechanic. The dynamax mechanic allows Pokémon to temporarily increase their power and HP, adding another layer of strategic complexity to battles.
D.1. Results without Dynamax
Table 4 presents the results for Gen Random Battles without the dynamax mechanic.
Table 4, Gen Random Battles without dynamax mechanic.
Win Rate
Bot Method
Language Model
Abyssal (%)
PokéChamp
GPTT-00
70
PokéChamp
Llama 3.1:8b
64
PokéLLMon (Hu et al., 2024b)
GPT-4o
56
One Step Lookahead
N/A
44
In this setting, PokéChamp with GPT-4o achieves a 70% win rate against the Abyssal bot, significantly outperforming other methods. Notably, PokéChamp using the smaller Llama 3.1:8b model still outperforms PokéLLMMon with GPT-4o, demonstrating the effectiveness of our approach even with less powerful language models.
D.2. Results with Dynamax
Table 5 shows the results for Gen 8 Random Battles with the dynamax mechanic enabled.
Table 5. Gen Random Battles with dynamax mechanic.
Win Rate vs.
Method
LLM
Elo
Avg. # Turns
Abyssal %
PokéChamp
GPT-4o
56
1273
17.1
PokéChamp
Llama 3.1:8b
52
1184
19.1
PokéL.LMon
GPT-4o
36
1048
22.5
Abyssal
N/A
N/A
1213
19.0
One Step Lookahead
N/A
16
998
18.9
Max Power
N/A
4
787
23.2
Random
N/A
0
493
24.3
With dynamax enabled, PokéChamp maintains its superior performance, achieving the highest Elo rating of 1273 and the shortest average game length of 17.1 turns. This indicates that PokéChamp can effectively utilize the dynamax mechanic to gain advantages and close out games more quickly.
D.3. Analysis of Results
Figure provides detailed view of the pairwise matchups between different methods in Gen Random Battles.
These results demonstrate several key points:
Adaptability: PokéChamp's strong performance in Random Battles highlights its ability to quickly assess and adapt to
17

---

PokéChamp: an Expert-level Minimax Language Agent
Model Winrate Matrix
ISeeCChmmp-
0.64
0.76
0.56
0.92
0.88
1.00
PortChamp ILlama)
0.36
0.64
0.52
0.72
0.92
1.00
Model
Abyssal
0.64
1.00
One Slep Lootatwwec
0.08
0.28
0.52
0.16
0.92
0.84
0.24
0.04
Random-
0.04
0.00
PaaeChamp
PokeChamp tlamal
PrkelLMon
Abyssa
step Look aiolaand
Randon
Hear
Opponent Model
Figure Gen 8 Random Battles matchup matrix per method.
unfamiliar team compositions, a crucial skill in this format.
Effective of dynamax: The maintained performance with dynamax enabled suggests that PokéChamp can effectively incorporate this mechanic into its strategy, likely due the LLM'' understanding of its impact the game state.
3. Efficiency: The shorter average game length for PokéChamp indicates that it can identify and execute winning strategies more quickly than other methods.
The consistent outperformance other methods, even with the added randomness of team composition, demonstrates the robustness of PokéChamp's 's decision- decision-making process.
Robustness to randomness:
5. LLM size trade-off:: While GPT-4o provides the best performance, the strong results with Llama 3.1: suggest favorable trade-off between model size and performance for resource-constrained applications.
These experiments in the Random Battles format further validate the versatility and effectiveness of PokéChamp, showing that its performance advantages extend beyond fixed team compositions to scenarios with higher uncertainty and variability.
18


Reviews:
[{"review_id": "RSO3vt3zur", "reviewer": "Reviewer_NRnt", "summary": "The authors introduce a novel RL agent that integrates and LLM into the tree-search process showing that their method can provide acceptable decisions in complex game states.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**More detailed model behavior analysis beyond Section C**: In addition to Section C, our paper provides the following analyses on our model/method with respect to the mechanics and strategies present in this game. In Section 4.3 (page 5-6), we present benchmark puzzles specifically designed to test PokéChamp's strategic decision-making abilities with special mechanics like Terastallization and Dynamax. Figure 5 illustrates how PokéChamp demonstrates understanding of complex type matchups and strategic mechanics usage rather than exploiting defects. Additionally, we evaluate against both bots and human players (Section 5.3), showing that our approach generalizes beyond potential weaknesses in other models.\n\n**probability distribution over possible moves**: We assess this in Table 1 (page 6), where we evaluate action prediction accuracy from Top-1 through Top-5 (ranking according to the action probabilities), effectively showing a distribution over moves compared to actual human actions.There may be multiple valid strategies at any point, which is why we include the Top-K metrics. The player prediction accuracy for PokéChamp varies between 26-30% for Top-1 and improves to 43-66% for Top-5 as Elo increases. We appreciate your suggestion to look at perplexity on human moves, which we can incorporate in future work.\n\n**the \"black box\" nature of using an LLM**: We have taken great care to make our approach transparent. As illustrated in Figure 2 (page 3), we explicitly replace three components of minimax search with LLM-based generations and clearly explain the contribution of each component. The LLM serves as a prior to constrain the search space to human-like strategies (page 2), leveraging general knowledge rather than Pokemon-specific training. Our ablation studies comparing PokéChamp with Llama 3.1 versus GPT-4o (Tables 3-4) demonstrate that some performance gains come from the intrinsic model capacity. However, we also provide ablations to show that all tested models greatly benefit from our methodology (comparing PokéChamp with PokéLLMon, Tables 3-4). In fact, being able to switch in the latest frontier model or open-source model when new capabilities emerge is an important benefit of this method.\n\nWe also demonstrate robustness across different formats (Gen 8 Random Battles, Gen 9 OU) and against human players (achieving 1500 Elo, top 10% of players), showing that our approach generalizes well beyond specific test environments.\n\nThank you for the positive feedback on our evaluation metrics and recognition of our work's relevance to both RL and LLM integration into task planning.", "claims_and_evidence": "The authors claim that their method is SOTA on pokemon which they test in multiple ways and against multiple other models. This is a clear and strong evidence for the claim.", "methods_and_evaluation_criteria": "Elo and winrate are not perfect metrics of RL search performance, but are the standard in these types of zero-sum games. So I don't have an issue with the evaluation. I would have liked some more detailed analysis of the model's behaviour (beyond section C), for example it might be exploiting defects in the other models, or taking very unusual lines against humans leading to an inflated winrate.\n\nCan the model output a probability distribution over possible moves (policy map)? Looking at perplexity on a dataset of human moves, or the sharpness of the distribution would reveal more about the underlying reasoning.", "theoretical_claims": "N/A, this is empirical work", "experimental_designs_or_analyses": "See above, and I am also concerned with the validity of using an LLM as a black box. This significantly reduces the generalizabilty of the results as we do not know how much of the uplift came from the LLM's prompting, or some components of the LLM's training data (i.e. there are many games online that might have been included).", "relation_to_broader_scientific_literature": "This is relevant to both RL and the larger project of integrating LLMs into task planning. As I said above the black box nature of the work is a sever limitation, but I think this helps highlight ways that LLMs can be used as a component of a larger planning system.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "See above", "other_comments_or_suggestions": "See above", "questions_for_authors": "See above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q4VfQ3tgvt", "reviewer": "Reviewer_yTKx", "summary": "This paper introduces PokeChamp, an LLM-powered game-theoretic agent designed for competitive Pokémon battles. PokeChamp uses LLM-guided minimax search to model decision-making in partially observable environments. It outperforms all prior LLM-based and heuristic-based Pokemon bots.\n\n## update after rebuttal\nThe authors address most of my concerns, and I keep my original rating to lean toward accepting the paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review. We appreciate your recognition of PokéChamp's strong performance against prior bots and human players.\n\n**Elo ratings and active players**: The Elo system on Pokémon Showdown (which we use for evaluation) only includes active players by design. Inactive accounts are reset to a base Elo of 1000 regularly and are not included in the percentile, ensuring our 90th percentile achievement reflects performance against the current active competitive community. As detailed in Figure 1 (page 1), PokéChamp's 1500 Elo rating places it firmly among expert players in this active ecosystem. This is comparable to how chess platforms like chess.com evaluate player performance. Pokemon Showdown has a smaller scale: roughly 3000 active games at any given time with millions of games played every month.\n\n**LLM overhead in real-time play**: We address this important constraint in Section 5.3 (page 7), noting that \"33% of games were lost due to time constraints.\" Our system implements strict time management to ensure compliance with the 15-second per turn limit, which restricts our search depth to a maximum of two-step lookahead with a branching factor of 16 (4 max player actions, 4 min player actions). In practice, the decision time varies based on game state complexity significantly faster when fewer Pokémon remain in play due to the reduced state space. We have further developed a faster version of this using better coding practices and systems techniques to increase speed that we will release with our code that achieves the same performance as mentioned in the paper.\n\n**Pokémon-specific terms**: We agree that better explanation of Pokémon-specific terms would improve accessibility. Elo is a standard rating system widely used in competitive gaming beyond Pokémon (originated in chess), and we define the Abyssal bot in Section 5.1 (page 6) as \"a rule-based heuristic bot used in official Pokémon games.\" We'll expand these definitions in the revised version.\n\n**Higher prediction accuracy at 1800+ Elo**: This interesting observation likely stems from the narrower strategy space employed by elite players. At this high level, players tend to favor optimal, established strategies rather than unpredictable or sub-optimal plays, making their decisions more consistent and thus more predictable.\n\n**Damage calculator**: The damage calculator example you noted (dragondarts: 161 turns) is actually a technical implementation detail. We cap the maximum turns to KO for cases where a Pokémon is effectively immune to an attack (as with Dragon-type moves against Fairy-types). By avoiding infinite values, we find that the LLM performs better at understanding and comparing all options. \n\n**Three LLM-based components**: Our three LLM-based components (action sampling, opponent modeling, and value estimation) form a general framework that is naturally applicable to any two-player partially observable game, not just Pokémon. These components could be adapted to other strategic games with similar characteristics.\n\n**Remaining comments**: We will address the missing ablation studies and hyperparameter sensitivity analysis description in our revised manuscript. We provide a link to a double-blind project website within our submitted project. The code link within the website is already announced “not for anonymous review” during the review period, but is available to reviewers after the review period is over. Thank you for your valuable feedback that will help us improve the final version.", "claims_and_evidence": "1. The overhead of LLM-based search is not fully discussed, particularly its impact on real-time play.\n\n2. While PokeChamp achieves the 90th percentile, there is a concern that a small fraction of players may be truly active. A comparison specifically with active players would provide a more realistic performance benchmark.", "methods_and_evaluation_criteria": "The paper introduces Pokemon battle-specific terms (e.g., \"Abyssal bot\", \"Elo\") without detailed explanation or references. This makes the work less accessible to researchers unfamiliar with Pokémon battles.", "theoretical_claims": "Not much theory is included in this paper.", "experimental_designs_or_analyses": "Sensitivity to hyperparameters (e.g., search depth) is not explored.\n\nAblation studies on the effectiveness of each component are missing.", "relation_to_broader_scientific_literature": "Missing some discussion about related papers. see \"Essential References Not Discussed\".", "essential_references_not_discussed": "The idea of integrating LLMs with the minimax search framework for game-playing agents is closely related to prior work by Guo et al. (2024), which explores a similar concept in two-player zero-sum games. Guo, Wei, et al. \"Minimax Tree of Thoughts: Playing Two-Player Zero-Sum Sequential Games with Large Language Models.\" ICML 2024 Workshop on LLMs and Cognition.", "other_strengths_and_weaknesses": "**Additional Weaknesses:**\n\nPrediction accuracy at higher Elo ratings: In Table 1, action prediction accuracy is highest at the 1800 Elo stage. Can the authors explain this phenomenon? Are skilled players' actions easier to predict?\n\nDamage calculator inconsistency: In Appendix line 685, \"dragondarts: 161 turns to KO Primarina\" seems incorrect, as Primarina is Fairy-type and should be immune to Dragon-type moves. How does the damage calculator work?", "other_comments_or_suggestions": "See weaknesses and questions.", "questions_for_authors": "1. Can you go deeper into the online evaluation score, such as Elo? While PokéChamp achieves expert-level performance (90th percentile Elo), how does it compare to active players?\n\n2. Can you provide the trend of decision time as the round progresses? At which step does the LLM’s response time exceed 15 seconds?\n\n3. PokeChamp implements three LLM-based components (action sampling, opponent modeling, value estimation). Are these components specifically designed for Pokemon battles, or do they or their intuition have broader applicability?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xERKDy0zyT", "reviewer": "Reviewer_734n", "summary": "The paper introduces PokéChamp, an agent that leverages minimax-based search to play competitive Pokemon battles. Specifically, the LLM performs action sampling, opponent modeling and state value calculation, allowing it to navigate partially observable state spaces of the battles. The authors also present various experiments in the paper showcasing that PokéChamp outperforms existing LLM-base and heuristic bots built for Pokemon battles.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for highlighting our framework's novelty and effectiveness in integrating minimax search with LLMs, and for recognizing PokéChamp's strong performance across multiple benchmarks.\n\n**generalizability beyond Pokémon battles**: Our framework naturally applies to any two-player zero-sum competitive games beyond Pokemon where minimax tree search is feasible. As described in Section 3 (page 3-4), our framework implements three LLM-powered components that can generalize to any POMG: \"(1) Action sampling via LLM and tool-assisted action generation, (2) Opponent modeling through historical data and LLM-based prediction, and (3) LLM-generated value function for leaf nodes. This method implements basic components of the minimax search tree with LLM predictions that can generalize to any POMG.\"\n\nThe core innovation here is the replacement of traditional minimax components with LLM-based alternatives. While we demonstrate this in Pokémon, the structure is applicable to any two-player partially observable game where states can be described in natural language. Unlike RL-based methods that require extensive task-specific training, our approach only requires observation space descriptions in natural language. This makes implementation no more difficult than developing custom RL environments and reward functions.\n\n**search depth trade-offs**: As detailed in Section 5.3 (page 7), we implement strict time management to comply with the 15-second per turn limit: \"Our search depth is limited by the 15 second time cutoff, which is a maximum of a two step lookahead with a branching factor of 16 (4 max player actions, 4 min player actions).\" We address this trade-off dynamically throughout gameplay, as \"the decision time when there is only 1 pokemon left is very fast due to the limited number of states to expand.\" This demonstrates our system's adaptability to different computational constraints.\n\n**opponent modeling accuracy**: While the raw prediction accuracies (13-16%) may appear low, this reflects the inherent challenge of predicting exact moves in a complex, partially observable environment with a large action space. Importantly, even imperfect opponent models provide significant value in minimax search by narrowing the branch exploration to more likely opponent responses. The comparative performances against state-of-the-art bots (76% win rate against the strongest LLM-based bot and 84% against the most advanced heuristic bot) demonstrate that our opponent modeling is effective despite these challenges.\n\nWe appreciate your overall positive assessment of our work and hope our clarifications address your concerns about generalizability and search depth optimization.", "claims_and_evidence": "The claims in the paper are largely well supported. For instance, many experiments were conducted and empirical results showed that PokéChamp outperforms existing rule-based bots and LLM-based agents under various different conditions when measuring win rates, with the measured Elo ratings also reinforce these results.\n\nHowever, the claim in the conclusion that this paper provides a generalized framework for other POMGs lack empirical evidence. The design of PokéChamp is highly specific to the mechanic of Pokemon battles, and hence while the framework is sound in theory, it is hard to tell how generalizable this method will be to other games.", "methods_and_evaluation_criteria": "The methods used do make sense for the application at hand. By integrating the three main minimax search steps: action sampling, opponent modeling and state value calculations, the agent can optimize decision making in the complex, partially observable environment. The one-step look ahead also helps improve decision making the estimating the immediate consequences of the potential actions, enabling PokéChamp to further improve the decisions made.\n\nThe evaluation that include comprehensive benchmarks against well known Pokemon bots and human players under diverse settings (which sometimes isolate particular game mechanics to test the agent’s understanding of those in particular) is robust. The metrics measured such as win rate and Elo are strong quantitative measures of the performance.", "theoretical_claims": "The paper formulates Pokemon battles as partially observable Markov game (POMGs), but there are no explicit theoretical claims of proofs. In general, there is a slight gap between the theoretical game theory framework and the LLM generated value functions used in PokéChamp, as it is unclear how certain theoretical properties translate when using LLM-based approximations of the values.", "experimental_designs_or_analyses": "The battles that are conducted in the experiments span multiple setups and compare multiple baselines bots and human players to PokéChamp, providing a comprehensive evaluation. The metrics such as win rate, Elo and average number of turns are suitable measures of the performance of the framework.\n\nThe analysis of the weak points of PokéChamp (such as losses due to time, or handling stall tactics and excessive switching) were insightful in identifying what the limitations are and where the framework could be improved.", "relation_to_broader_scientific_literature": "The paper builds on the idea of developing AI that achieves superhuman performance in games, it draws reinforcement learning approaches from work such as AlphaZero, and extend those ideas by integrating LLMs into the framework to reduces the expensive training required in the existing frameworks. It also improves the research on other LLM based agents such as PokéLLMon by incorporating these planning algorithms. The combination of the two methodologies introduces ideas that can be used in further research, and are especially practical due to the lack of extensive training requirements.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n- The paper presents a novel framework to integrate minimax search algorithms with LLM agents, enabling improved performance without the massive training requirements\n- PokéChamp outperforms existing bots and agents across many benchmarks and battle configurations, proving its efficacy\n\nWeaknesses:\n- The biggest weakness of this paper is the limited generalization it has to other applications. Many components of PokéChamp are highly specific to Pokemon limits the ability for this framework to be easily applicable to other games without significant additional configurations (unlike more general RL based methods that can be generalized with less effort)\n- Opponent modeling is mentioned as a key component of the framework, however the prediction accuracies are very low (13-16%) and hence more analysis into this step would be helpful to investigate the effectiveness of it", "other_comments_or_suggestions": "N/A", "questions_for_authors": "- Can the authors speak more about the trade-offs associated with search depth and what their approaches are to optimize this balance?\n- Can the authors speak more generally about the generalization of these methods beyond Pokemon battles?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fK2MVDd7zE", "reviewer": "Reviewer_UL1n", "summary": "This paper introduces PokeChamp, and LLM combined with a game-playing agent to perform minimax search for winning Pokemon battles. The authors replace several parts of minimax search with an LLM and introduce a Pokemon battling dataset to understand LLM agent's failures. PokeChamp is able to reach the top 90% of human performnace.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the value of our exploration of LLM agents for partially observable games and appreciating our dataset contribution. These are indeed core strengths of our work.\n\n**LLM utilization**: LLMs are key components in our system to achieve the claimed performance. Our work represents an important attempt to effectively utilizing LLMs in complex two-player game environments: While opponent stat estimation could potentially be learned from data alone, LLMs provide crucial domain knowledge that would otherwise require extensive labeled data and model training. For instance, they understand type matchups, move effectiveness, and Pokemon meta-strategies with minimal prompting (as shown in Figure 5, page 6, where PokéChamp adapts to complex mechanics like Terastallization and Dynamax). The one-step lookahead mechanism combines LLM knowledge with damage calculations to determine the best action for the game by extrapolating over short horizon information.\n\nWhile opponent action prediction is challenging (13-16% accuracy for min-player, Table 1, page 5), this is still valuable for narrowing the search space in minimax and substantially better than random (which would be <1% given the large action space). Even imperfect opponent modeling significantly improves overall performance, as demonstrated by our 76% win rate against the strongest LLM-based bot and 84% against the strongest heuristic bot (Table 3, page 7).\n\n**Pokemon as illustrative settings for LLM**: Pokémon battle is a text-based two-player game featuring a vast pool of Pokémon, types, items, moves, and game mechanics, all described in text, resulting in a combinatorial explosion of possible configurations. It exemplifies a complex, partially observable environment with a massive state space (~10^354 for the first turn alone, page 2). This complexity makes it an excellent testbed for demonstrating how LLMs can constrain search to human-like strategies without explicit training, which is our core contribution.\n\n**ablation studies**: We appreciate this suggestion. We do provide a partial ablation by comparing PokéChamp with GPT-4o versus Llama 3.1:8b (Tables 3-4, page 7-8), demonstrating that our approach works well even with smaller models. Though frontier models such as GPT-4o perform best. We also compare against the One Step Lookahead agent without the full minimax framework (Tables 3-4), showing the value of our complete approach.\n\nRegarding the value function quality, as mentioned in our initial response, the best evaluation is performance in games. Our expert-level performance (1500 Elo rating, top 10%, Figure 1, page 1) demonstrates the effectiveness of our value function approximation. The value function is able to correctly prioritize actions that lead to winning strategies, as shown by our significantly higher win rates compared to all baselines.\n\n**the significance of 90% Elo**: Pokémon Showdown is a very active competitive ladder with roughly 3000 active games at any given time, and over millions of games occurring every month. This makes the top 10% achievement substantial. Reaching this level requires deep understanding of complex strategies, team compositions, and meta-game knowledge. The 1500 Elo rating on Pokémon Showdown is considered expert-level play among the community.\n\nWe thank you for your suggestions regarding model naming conventions and will address these in our revision.", "claims_and_evidence": "See strengths and weaknesses", "methods_and_evaluation_criteria": "See strengths and weaknesses", "theoretical_claims": "See strengths and weaknesses", "experimental_designs_or_analyses": "See strengths and weaknesses", "relation_to_broader_scientific_literature": "See strengths and weaknesses", "essential_references_not_discussed": "See strengths and weaknesses", "other_strengths_and_weaknesses": "# Strengths\n- The paper provides an interesting exploration of how to use LLM agents for partially observable games. There are several ideas that were explored that could point towards future paths to understanding how to mix LLMs with more classical AI systems.\n- It is clear that the authors enjoyed creating their agent, which makes the paper fun to read. \n- The authors collected a dataset which will greatly help enable progress towards studying partially observable games. Moreover, the authors use the dataset to quantify their agent's performance.\n\n# Weaknesses\n- **The LLM does not appear to be well utilized:** \n    - The LLM appears in the one-step lookahead when estimating the opponent's stats, the opponent's action prediction, and the value function. In the first case, it appears that the estimating the opponent's stats from data likely can be learned from data and does not require a language prior. In the second case, the authors show that the LLM's performance is slightly above random (although this could possibly be due to Pokemon data being OOD for the LLM), and in the third case there is no evidence that the LLM's estimation of the value function is actually accurate (although I realize that this is difficult to evaluate). \n    - While I think this work is still interesting as an exploration of how to combine LLMs with agents, I don't think the Pokemon setting is particularly illustrative for showcasing the strengths of the LLM. Much of this work could be done with standard DNNs. \n- No ablation studies of the agent: would it be possible for the authors to ablate some of the LLM components of the agents to see where the performance gains are coming from? Some suggestions: a) replace the LLM in the stat estimation with a trained NN, b) remove the LLM from the action prediction (randomly predict an action or train a NN to predict the action), or c) use smaller LLMs, e.g., Llama-3.2-3B. How much does this tank performance?\n- No studies of the agent's value function: although I realize that it's extremely difficult to even empirically estimate the value function, could the authors speak to the quality of the estimated value function produced by th eLLM?", "other_comments_or_suggestions": "Nit: change GPT 4-o to 4o everywhere (I noticed this in the abstract) and change Llama3.1:8b to Llama-3.1-8B (L320). Also are you using the instruct version? If so that should be mentioned.", "questions_for_authors": "Is 90% ELO good in an absolute sense? I wonder if there's just not many people playing Pokemon battles or the average skill level of human players is pretty low. I'm guessing it would take someone much more time to get 90% at chess than at Pokemon battles. Do you have a sense of the difficulty of the accomplishment? (I still think it's a cool result regardless but it's a bit abstract for me).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents an innovative agent, PokéChamp, that integrates LLM-driven components into a minimax tree search to address the challenges of competitive Pokémon battles with partially observable states. The approach effectively combines action sampling, opponent modeling, and state value evaluation, leading to impressive performance improvements over traditional heuristic and LLM-based bots, as evidenced by robust win rate and Elo rating benchmarks. Although the empirical results within the Pokémon domain are convincing, the broader claim regarding the generalizability of the framework to other partially observable, two-player zero-sum games remains more speculative. Overall, the submission is a well-executed blend of classical AI and modern language model techniques, contributing a valuable dataset and novel insights.
Decision: Accept (spotlight poster)