MolmoAct: Action Reasoning Models that can Reason in Space
Authors: name: Jason Lee, name: Jiafei Duan, name: Haoquan Fang, name: Yuquan Deng, name: Shuo Liu, name: Boyang Li, name: Bohan Fang, name: Jieyu Zhang, name: Yi Ru Wang, name: Sangho Lee, name: Winson Han, name: Wilbert Pumacay, name: Angelica Wu, name: Rose Hendrix, name: Karen Farley, name: Eli VanderBilt, name: Ali Farhadi, name: Dieter Fox, name: Ranjay Krishna
Organization: Ai2

Reasoning is central to purposeful action, yet most robotic foundation models
map perception and instructions directly to control, which limits adaptability,
generalization, and semantic grounding. We introduce Action Reasoning Models
(ARMs), a class of vision-language-action models that integrate perception,
planning, and control through a structured three-stage pipeline. Our model,
MolmoAct, encodes observations and instructions into depth-aware perception
tokens, generates mid-level spatial plans as editable trajectory traces, and
predicts precise low-level actions, enabling explainable and steerable
behavior. MolmoAct-7B-D achieves strong performance across simulation and
real-world settings: 70.5% zero-shot accuracy on SimplerEnv Visual Matching
tasks, surpassing closed-source Pi-0 and GR00T N1; 86.6% average success on
LIBERO, including an additional 6.3% gain over ThinkAct on long-horizon tasks;
and in real-world fine-tuning, an additional 10% (single-arm) and an additional
22.7% (bimanual) task progression over Pi-0-FAST. It also outperforms baselines
by an additional 23.3% on out-of-distribution generalization and achieves top
human-preference scores for open-ended instruction following and trajectory
steering. Furthermore, we release, for the first time, the MolmoAct Dataset --
a mid-training robot dataset comprising over 10,000 high quality robot
trajectories across diverse scenarios and tasks. Training with this dataset
yields an average 5.5% improvement in general performance over the base model.
We release all model weights, training code, our collected dataset, and our
action reasoning dataset, establishing MolmoAct as both a state-of-the-art
robotics foundation model and an open blueprint for building ARMs that
transform perception into purposeful action through structured reasoning.
Blogpost: https://allenai.org/blog/molmoact