EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation
Authors: name: Siyuan Huang; name: Liliang Chen; name: Pengfei Zhou; name: Shengcong Chen; name: Zhengkai Jiang; name: Yue Hu; name: Peng Gao; name: Hongsheng Li; name: Maoqing Yao; name: Guanghui Ren
Published: 2025-01-03 17:00:33

Abstract: We introduce EnerVerse, a comprehensive framework for embodied future space
generation specifically designed for robotic manipulation tasks. EnerVerse
seamlessly integrates convolutional and bidirectional attention mechanisms for
inner-chunk space modeling, ensuring low-level consistency and continuity.
Recognizing the inherent redundancy in video data, we propose a sparse memory
context combined with a chunkwise unidirectional generative paradigm to enable
the generation of infinitely long sequences. To further augment robotic
capabilities, we introduce the Free Anchor View (FAV) space, which provides
flexible perspectives to enhance observation and analysis. The FAV space
mitigates motion modeling ambiguity, removes physical constraints in confined
environments, and significantly improves the robot's generalization and
adaptability across various tasks and settings. To address the prohibitive
costs and labor intensity of acquiring multi-camera observations, we present a
data engine pipeline that integrates a generative model with 4D Gaussian
Splatting (4DGS). This pipeline leverages the generative model's robust
generalization capabilities and the spatial constraints provided by 4DGS,
enabling an iterative enhancement of data quality and diversity, thus creating
a data flywheel effect that effectively narrows the sim-to-real gap. Finally,
our experiments demonstrate that the embodied future space generation prior
substantially enhances policy predictive capabilities, resulting in improved
overall performance, particularly in long-range robotic manipulation tasks.