we-dream/pretrain-action-free-2kh

WE-Dream action-free pretraining — 2kh

This checkpoint is the action-free pretraining baseline of WE-Dream, saved at step 50,000 using the 2,000-hour EgoDex + EgoVerse pretraining setup. Training uses video prediction without action supervision: the saved training transform sets action values and their loss mask to zero, and marks samples as having no real action. The action-prediction loss is therefore zero. The architecture still contains a 32-dimensional action interface and a one-dimensional placeholder state; this checkpoint should not be described as trained with 32-D latent-action targets.

The complete package preserves all 19 original safetensors shards (91.69 GB), including the DiT, text encoder, image encoder, and VAE. The saved tensors are predominantly FP32, with a small BF16 subset in the VAE; no precision conversion or component removal is applied. The configured horizon is 24 and the routed-attention probability is 0.5.

Usage

Use the WE-Dream codebase and its environment and base-checkpoint setup. These weights require the project's custom model classes; this is not a standalone Transformers pipeline. Download the Wan auxiliary components described in that guide. The saved transforms use the google/umt5-xxl tokenizer.

from huggingface_hub import snapshot_download

checkpoint_dir = snapshot_download(
    repo_id="we-dream/pretrain-action-free-2kh",
    local_dir="checkpoints/pretrain-action-free-2kh",
)

If access is gated, request access on this page and sign in with hf auth login. Set revision to the release commit for reproducible downloads.

Use this checkpoint as PRETRAINED_MODEL_PATH for robot post-training. It is a pretraining checkpoint, not a robot controller.

Package metadata

Model targets and resource paths are made portable. The original normalization metadata and weight index are preserved; experiment_cfg/conf.yaml contains the runtime fields, transforms, and modality definitions for the three source embodiments. Training-environment settings and machine-specific paths are omitted. See provenance.json for the source checkpoint step, file sizes, header fingerprints, and metadata changes. Full weight SHA256 values are available in the published Hugging Face file metadata after release verification.