Dita: Scaling Diffusion Transformer for Generalist
  Vision-Language-Action Policy
Authors: name: Zhi Hou; name: Tianyi Zhang; name: Yuwen Xiong; name: Haonan Duan; name: Hengjun Pu; name: Ronglei Tong; name: Chengyang Zhao; name: Xizhou Zhu; name: Yu Qiao; name: Jifeng Dai; name: Yuntao Chen
Published: 2025-03-25 15:19:56

Abstract: While recent vision-language-action models trained on diverse robot datasets
exhibit promising generalization capabilities with limited in-domain data,
their reliance on compact action heads to predict discretized or continuous
actions constrains adaptability to heterogeneous action spaces. We present
Dita, a scalable framework that leverages Transformer architectures to directly
denoise continuous action sequences through a unified multimodal diffusion
process. Departing from prior methods that condition denoising on fused
embeddings via shallow networks, Dita employs in-context conditioning --
enabling fine-grained alignment between denoised actions and raw visual tokens
from historical observations. This design explicitly models action deltas and
environmental nuances. By scaling the diffusion action denoiser alongside the
Transformer's scalability, Dita effectively integrates cross-embodiment
datasets across diverse camera perspectives, observation scenes, tasks, and
action spaces. Such synergy enhances robustness against various variances and
facilitates the successful execution of long-horizon tasks. Evaluations across
extensive benchmarks demonstrate state-of-the-art or comparative performance in
simulation. Notably, Dita achieves robust real-world adaptation to
environmental variances and complex long-horizon tasks through 10-shot
finetuning, using only third-person camera inputs. The architecture establishes
a versatile, lightweight and open-source baseline for generalist robot policy
learning. Project Page: https://robodita.github.io.