Jaber628/LD4WAM_WDAM_pretrain

LD4WAM — WDAM pretrained checkpoint

Pretrained World Dynamics Action Model (WDAM) from LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models.

The model couples the Wan2.2-TI2V-5B video expert, a latent-dynamics expert supervised by the frozen LDM (Jaber628/LD4WAM_LDM), and an action expert (20-D end-effector actions with proprioception, 3-view observation canvas). It is the starting point for task-specific post-training, e.g. on RoboTwin 2.0, with the code in the LD4WAM repository (WDAM/).

Files

File	Description
checkpoint_step_9415.safetensors	full model weights (bf16 training)
config.yaml	model architecture config; replace the <...> placeholders with local paths
tokenizer/	UMT5 tokenizer used by the Wan text encoder (kept with the checkpoint so deployment is self-contained)

Usage

Post-train from this checkpoint with the WDAM code:

cd LD4WAM/WDAM
NPROC_PER_NODE=8 bash scripts/train.sh \
    training.finetune_ckpt_path=/path/to/LD4WAM_WDAM_pretrain \
    model.video_backbone.model_path=/path/to/Wan2.2-TI2V-5B \
    model.architecture.latent_action_model.ckpt=/path/to/ldm_model_final.pt \
    dataloader.dataset_dir=/path/to/RoboTwin2.0/dataset \
    training.output_path=/path/to/ckpts

Or serve it directly as a policy:

bash scripts/deploy.sh /path/to/LD4WAM_WDAM_pretrain --port 8848

See the repository README for environment setup, evaluation and the full option list.

Citation

@misc{shen2026ld4wamlearninglatentdynamics,
      title={LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models},
      author={Zhenhao Shen and Jiaqi Liang and Jasper Lu and Feng Jiang and Yuran Wang and Chuanbo Wei and Jiayi Liu and Jianchun Yang and Qize Yu and Jiadi You and Ce Hao and Guanqi He and Chen Xie and Ruihai Wu},
      year={2026},
      eprint={2608.22403},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2608.22403},
}