ICML 2025

Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination
Kunal Jha 1 Wilka Carvalho 2 Yancheng Liang 1 Simon S. Du 1 Max Kleiman-Weiner * 1 Natasha Jaques * 1
Abstract
Zero-shot coordination (ZSC), the ability to adapt
to a new partner in a cooperative task, is a criti-
cal component of human-compatible AI. While
prior work has focused on training agents to co-
operate on a single task, these specialized mod-
els do not generalize to new tasks, even if they
are highly similar.
Here, we study how rein-
forcement learning on a distribution of environ-
ments with a single partner enables learning
general cooperative skills that support ZSC with
many new partners on many new problems.
We introduce two Jax-based, procedural genera-
tors that create billions of solvable coordination
challenges. We develop a new paradigm called
Cross-Environment Cooperation (CEC), and
show that it outperforms competitive baselines
quantitatively and qualitatively when collaborat-
ing with real people. Our findings suggest that
learning to collaborate across many unique scenar-
ios encourages agents to develop general norms,
which prove effective for collaboration with dif-
ferent partners. Together, our results suggest a
new route toward designing generalist coopera-
tive agents capable of interacting with humans
without requiring human data. Code for environ-
ment, training, and testing scripts and more can
be found at https://kjha02.github.io/
publication/cross-env-coop.
1. Introduction
Humans excel at ad-hoc cooperation, readily adapting to
new partners and environments by attending jointly to rel-
evant objects, reasoning about shared intentions, and play-
ing their role within an implicit collective plan (Tomasello,
*Equal contribution
1Department of Computer Science, Uni-
versity of Washington, Seattle, WA 2Kempner Institute for the
Study of Natural and Artificial Intelligence, Harvard Univer-
sity, Cambridge, Massachusetts. Correspondence to: Kunal Jha
<kjha@uw.edu>....