OpenWAM-Stanford/OpenWAM-Pretraining

OpenWAM Pretraining

This is an OpenWAM video pretraining checkpoint. It is an intermediate research checkpoint for text-conditioned latent video prediction, trained with single-view video and synchronized multiview mosaics. It is intended for video prediction research and subsequent adaptation; it is not an action-finetuned robot deployment policy.

Project: OpenWAM.

Released weights

File	Purpose	Size
model_state.pt	Model weights for inference or initialization	20,433,187,287 bytes
resolved_config.yaml	Saved experiment configuration for reference	14,425 bytes
SHA256SUMS	Integrity hashes for the weights and config	—
release_metadata.json	Source checkpoint provenance and release manifest	—

This is a weights-only release. Optimizer, scheduler, and resumable trainer states are not included. The original model weight file is preserved byte-for-byte.

Training stage

• Objective: latent video prediction, with task text conditioning.
• Architecture: OpenWAM/VPM causal video prediction pipeline; a 30-layer transformer with hidden size 3,072, 24 attention heads, and 48-channel Wan 2.2 VAE latents.
• Global batch size: 288 (36 per GPU × 8 GPUs, accumulation 1).
• Variable-length batching: bucket mode, with bucket pool size 1,152.
• Optimizer: AdamW; learning rate 1e-5 in this phase.
• Checkpoint interval: 1,000 optimizer steps.

The frozen data pool for this phase contained 1,053,709 single-view clip entries and 107,317 multiview clip entries. This describes the available pool, not a claim that every entry was consumed. Single-view sources include LIBERO, UMI, AgiBot, RoboMind, InternData, RoboCOIN, FastUMI, EgoExo4D RGB, and EgoExo4D SLAM. Multiview augmentation is added alongside single views; EgoExo4D RGB and SLAM are excluded from this augmentation.

During the final 1,000-step phase, the eight ranks recorded 288,000 training window presentations: 278,314 single-view and 9,686 multiview. These counts include repeated sampling and are not unique episode counts or the cumulative exposure over all earlier training stages. Data encoded or admitted after this phase is not included in the figures above.

Multiview inputs combine synchronized cameras before VAE encoding while preserving each view's aspect ratio. Current layouts include horizontal pairs for LIBERO and FastUMI, and head-over-wrists arrangements where applicable. The six-camera RoboMind subset uses camera_top, rotated 180 degrees, above the two wrist views. FastUMI retains full image height with limited outer black-border trimming.

Download and loading

from huggingface_hub import snapshot_download

release_dir = snapshot_download(
    repo_id="OpenWAM-Stanford/OpenWAM-Pretraining",
    local_dir="OpenWAM-Pretraining",
)

The native model payload can be read with PyTorch:

from pathlib import Path
import torch

payload = torch.load(
    Path(release_dir) / "model_state.pt",
    map_location="cpu",
    mmap=True,
    weights_only=True,
)
model_state_dict = payload["model_state_dict"]

Instantiate the matching OpenWAM/VPM pipeline before loading the state dict. This release uses the native model-state format and is not a standalone AutoModel.from_pretrained or Diffusers pipeline export. The saved config contains training-cluster paths; external VAE, text encoder, tokenizer, and data/cache paths must be supplied for your environment. Those external assets and training data are not bundled in this weights repository.

Check download integrity from the release directory with sha256sum -c SHA256SUMS.

Evaluation and limitations

Visual checks are limited, and lower MSE does not imply uniformly better visual quality. Long forecasts can stagnate, drift, deform objects, or blur details. RoboMind and RoboCOIN examples do not yet reliably follow the ground-truth manipulation. This checkpoint should be interpreted as an intermediate OpenWAM pretraining checkpoint.