J-aso-n/P-WAM-RoboCasa

P-WAM RoboCasa

Model checkpoint for P-WAM: Perception-World-Action Models on the RoboCasa benchmark.

P-WAM is a framework that enhances World Action Models (WAMs) with pretrained visual perception encoders while preserving their world dynamics modeling. It introduces complementary semantic and spatial perceptual features from SigLIP and DINOv3 into the video-action DiT through lightweight connector modules.

Checkpoint contents

This PyTorch Distributed Checkpoint (DCP) includes the DiT, fine-tuned visual encoders, feature fuser, connector, and context alignment parameters.

model/
  .metadata
  __0_0.distcp

Keep both files together with their original names.

Usage

Follow ROBOCASA.md for environment setup and evaluation. The matching model configuration is included in the P-WAM code.

In the evaluation command, use:

--config_file cosmos_policy/config/config.py
--config cosmos_predict2_2b_480p_robocasa_50_demos_per_task__vision_condition__inference
--ckpt_path J-aso-n/P-WAM-RoboCasa

The evaluation entry point downloads this repository and selects its model/ directory. You can also download it manually:

hf download J-aso-n/P-WAM-RoboCasa --local-dir models/P-WAM-RoboCasa

For a local download, set --ckpt_path ./models/P-WAM-RoboCasa in the evaluation command.

Evaluation also requires the VAE tokenizer, local SigLIP and DINOv3 model directories, RoboCasa normalization statistics, and T5 text embeddings. Prepare these assets as described in the code repository's RoboCasa instructions.

License

Built on NVIDIA Cosmos.

The Cosmos-derived checkpoint is subject to the NVIDIA One-Way Noncommercial License (NSCLv1), including its noncommercial-use restriction. The fine-tuned visual encoders retain their upstream terms:

• DINOv3: DINOv3 License.
• SigLIP: Apache License 2.0.

See NOTICE for component provenance and attribution.