EnerVerse-AC: Envisioning Embodied Environments with Action Condition
Authors: name: Yuxin Jiang; name: Shengcong Chen; name: Siyuan Huang; name: Liliang Chen; name: Pengfei Zhou; name: Yue Liao; name: Xindong He; name: Chiming Liu; name: Hongsheng Li; name: Maoqing Yao; name: Guanghui Ren
Published: 2025-05-14 18:30:53

Abstract: Robotic imitation learning has advanced from solving static tasks to
addressing dynamic interaction scenarios, but testing and evaluation remain
costly and challenging due to the need for real-time interaction with dynamic
environments. We propose EnerVerse-AC (EVAC), an action-conditional world model
that generates future visual observations based on an agent's predicted
actions, enabling realistic and controllable robotic inference. Building on
prior architectures, EVAC introduces a multi-level action-conditioning
mechanism and ray map encoding for dynamic multi-view image generation while
expanding training data with diverse failure trajectories to improve
generalization. As both a data engine and evaluator, EVAC augments
human-collected trajectories into diverse datasets and generates realistic,
action-conditioned video observations for policy testing, eliminating the need
for physical robots or complex simulations. This approach significantly reduces
costs while maintaining high fidelity in robotic manipulation evaluation.
Extensive experiments validate the effectiveness of our method. Code,
checkpoints, and datasets can be found at
<https://annaj2178.github.io/EnerverseAC.github.io>.