NeurIPS 2025

Imagined Autocurricula
Ahmet H. Güzel∗
University College London AI Centre
Matthew T. Jackson
University of Oxford
Jarek L. Liesen
University of Oxford
Tim Rocktäschel
University College London AI Centre
Jakob N. Foerster
University of Oxford
Ilija Bogunovic
University College London AI Centre
Jack Parker-Holder
University College London AI Centre
Abstract
Training agents to act in embodied environments typically requires vast training
data or access to accurate simulation, neither of which exists for many cases in
the real world. Instead, world models are emerging as an alternative–leveraging
offline, passively collected data, they make it possible to generate diverse worlds
for training agents in simulation. In this work, we harness world models to gen-
erate “imagined” environments to train robust agents capable of generalizing to
novel task variations. One of the challenges in doing this is ensuring the agent
trains on useful generated data. We thus propose a novel approach IMAC (Imag-
ined Autocurricula) leveraging Unsupervised Environment Design (UED), which
induces an automatic curriculum over generated worlds. In a series of challeng-
ing, procedurally generated environments, we show it is possible to achieve strong
transfer performance on held-out environments having trained only inside a world
model learned from a narrower dataset. We believe this opens the path to utilizing
larger-scale, foundation world models for generally capable agents.
1
Introduction
Despite significant advancements in AI, we remain far from generally capable agents [1]. In the
2010s, progress towards this goal was driven by breakthroughs in deep reinforcement learning (RL,
[2]) where agents such as AlphaGo [3] showed it was possible to discover new knowledge beyond
human capabilities, like the famous move37. However, scaling RL in simulation to produce a more
general intelligence remains bottlenecked by the lack of sufficiently diverse and real-world simula-
tors [4, 5]. By contrast...