oucliuyang/GAWM-S-RoboTwin2

GAWM-S for RoboTwin 2.0

GAWM (Geometry-Aware World Model) is a compact multi-task world-action model. A lightweight predictor forecasts the patch features of a frozen DINOv3 ViT-S/16 encoder at two future horizons, and an action decoder regresses action chunks from the current and forecast tokens. GAWM uses neither a vision-language model nor a video generator.

This repository holds the RoboTwin 2.0 model reported in the paper.

Results

Mixed protocol: one policy for all 50 tasks; 100 episodes per task in each setting (5,000 episodes), seed 0.

Setting	Success rate	95% interval (Wilson)
Clean	93.94% (4,697 / 5,000)	[93.24, 94.57]
Randomized	93.10% (4,655 / 5,000)	[92.36, 93.77]

Model

Parameters	77.0M in total: DINOv3 ViT-S/16 encoder 21.6M (frozen, not in this repository) and adapter, predictor, task code and action decoder 55.4M (this checkpoint)
Inputs	head and front cameras at 320×240, 16-D end-effector state, and the name of one of the 50 tasks, which selects its task code stored in the checkpoint
Output	32 steps of 14-D absolute joint targets; the evaluation executes the first 16 after cubic B-spline smoothing
Latency	8.8 ms per action-chunk query (median of 200; batch size 1, bf16, NVIDIA RTX PRO 5000; from preprocessed inputs to the action chunk)

Files

File	Content
checkpoints/GAWM-S-RoboTwin2.safetensors	weights
config.yaml	model configuration
dataset_statistics.json	action and state normalization
SHA256SUMS	checksums of the files above

Usage

The model runs with the GAWM code at github.com/gaoxiang12/gawm-public; set up its environment as described in the README, then from the repository root:

hf download oucliuyang/GAWM-S-RoboTwin2 --local-dir playground/Checkpoints/GAWM-S-RoboTwin2
(cd playground/Checkpoints/GAWM-S-RoboTwin2 && sha256sum -c SHA256SUMS)

DINOv3 ViT-S/16 is downloaded to playground/Pretrained on first use. Loading logs a warning that the keys backbone.encoder.* and world_model.feature_decoder.* are missing. This is expected: the encoder weights come from the official DINOv3 release, and the feature decoder is not used at inference.

Closed-loop evaluation with the protocol above (use --task-config demo_randomized and a new output directory for the randomized setting):

python scripts/run_robotwin_hdf5_clean_eval.py \
  --checkpoint playground/Checkpoints/GAWM-S-RoboTwin2/checkpoints/GAWM-S-RoboTwin2.safetensors \
  --output playground/Evaluations/gawm_s_robotwin2_clean100 \
  --tasks all --episodes 100 --gpus 0 1 2 3 4 5 6 7 --seed 0 \
  --task-config demo_clean --image-channel-order rgb

License

MIT. DINOv3 is not included; it is distributed by Meta under the DINOv3 License.

Citation

The paper will be linked here once it is public.