ICML 2025

PokéChamp: an Expert-level Minimax Language Agent
Seth Karten * 1 Andy Luu Nguyen 1 Chi Jin 1
Abstract
We introduce PokéChamp, a minimax agent
powered by Large Language Models (LLMs) for
Pokémon battles. Built on a general framework
for two-player competitive games, PokéChamp
leverages the generalist capabilities of LLMs to
enhance minimax tree search. Specifically, LLMs
replace three key modules: (1) player action sam-
pling, (2) opponent modeling, and (3) value func-
tion estimation, enabling the agent to effectively
utilize gameplay history and human knowledge to
reduce the search space and address partial observ-
ability. Notably, our framework requires no addi-
tional LLM training. We evaluate PokéChamp
in the popular Gen 9 OU format. When powered
by GPT-4o, it achieves a win rate of 76% against
the best existing LLM-based bot and 84% against
the strongest rule-based bot, demonstrating its su-
perior performance. Even with an open-source 8-
billion-parameter Llama 3.1 model, PokéChamp
consistently outperforms the previous best LLM-
based bot, Pokéllmon powered by GPT-4o, with
a 64% win rate. PokéChamp attains a projected
Elo of 1300-1500 on the Pokémon Showdown
online ladder, placing it among the top 30%-10%
of human players. In addition, this work compiles
the largest real-player Pokémon battle dataset, fea-
turing over 3 million games, including more than
500k high-Elo matches. Based on this dataset,
we establish a series of battle benchmarks and
puzzles to evaluate specific battling skills. We fur-
ther provide key updates to the local game engine.
This work establishes Pokémon as a benchmark to
integrate LLM technologies with game-theoretic
algorithms addressing general multi-agent prob-
lems. Videos, code, and dataset are available on-
line.
*Equal contribution 1Princeton University. Correspondence to:
Seth Karten <sethkarten@princeton.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright ...