Emma-X: An Embodied Multimodal Action Model with Grounded Chain of
  Thought and Look-ahead Spatial Reasoning
Authors: name: Qi Sun; name: Pengfei Hong; name: Tej Deep Pala; name: Vernon Toh; name: U-Xuan Tan; name: Deepanway Ghosal; name: Soujanya Poria
Published: 2024-12-16 16:58:28

Abstract: Traditional reinforcement learning-based robotic control methods are often
task-specific and fail to generalize across diverse environments or unseen
objects and instructions. Visual Language Models (VLMs) demonstrate strong
scene understanding and planning capabilities but lack the ability to generate
actionable policies tailored to specific robotic embodiments. To address this,
Visual-Language-Action (VLA) models have emerged, yet they face challenges in
long-horizon spatial reasoning and grounded task planning. In this work, we
propose the Embodied Multimodal Action Model with Grounded Chain of Thought and
Look-ahead Spatial Reasoning, Emma-X. Emma-X leverages our constructed
hierarchical embodiment dataset based on BridgeV2, containing 60,000 robot
manipulation trajectories auto-annotated with grounded task reasoning and
spatial guidance. Additionally, we introduce a trajectory segmentation strategy
based on gripper states and motion trajectories, which can help mitigate
hallucination in grounding subtask reasoning generation. Experimental results
demonstrate that Emma-X achieves superior performance over competitive
baselines, particularly in real-world robotic tasks requiring spatial
reasoning.