Spatial-UMI-Dream

Checkpoints of the Spatial-UMI-Dream world action model: a causal Wan2.2-TI2V-5B video DiT that jointly predicts future RGB + depth frames and end-effector actions.

Folder	Contents
pretrain/	Pretrained world action model (DiT weights only), the starting point for finetuning
robotwin-umi/checkpoint-10000/	Fine-tuned on the 20 RoboTwin-UMI tasks (50 clean Aloha-AgileX demonstrations per task, 10k steps)

Files

Each checkpoint directory holds config.json, model.safetensors.index.json and the safetensors shards; robotwin-umi/checkpoint-10000/experiment_cfg/ also holds the normalization statistics and the training config used at inference. Only the DiT (dit.*, 5.6B parameters, bf16) is stored. The frozen umt5-xxl text encoder, CLIP image encoder and Wan2.2 VAE are the unmodified official weights and are downloaded from Wan-AI/Wan2.2-TI2V-5B and Wan-AI/Wan2.1-I2V-14B-480P on first use.

Usage

Download the checkpoints into the Spatial-UMI-Dream code repository:

huggingface-cli download Yewandou/spatial-umi-dream --local-dir checkpoints/spatial-umi-dream

Fine-tune from checkpoints/spatial-umi-dream/pretrain, or evaluate checkpoints/spatial-umi-dream/robotwin-umi/checkpoint-10000 on RoboTwin-UMI, by following the README of the code repository.

License

Apache-2.0. The Wan components are released by Wan-AI under Apache-2.0.