SleepMastger/dish-utensil-flashwam

Dish-utensil Flashwam

Public bundle for the dish-utensil manipulation model trained by George on the SleepMastger/dish-utensil-manipulation dataset (100 teleoperated Franka episodes: 50 fork + 50 spoon, 10 Hz, 42,863 frames).

Status

This is the complete run: all 30 epochs (40,200 steps), checkpointed every 5 epochs. The model has not yet been evaluated on a robot.

Architecture: FlashWAM M1 fused-KV, fixed-RoPE, 1-layer action expert. Trained FROM SCRATCH (no LIBERO resume): Wan2.2 base video expert + ActionDiT linear-interp init, 30 epochs, global batch 32, cosine LR 1e-4, bf16 DeepSpeed ZeRO-1 on 4x GPU.

Checkpoints follow the same layout as the existing WAM repositories:

• checkpoints/weights/step_006700.pt — SHA-256 2787243d1dbdde220ca95974490af0c6906db574bfea5275cef641c1246f3e50
• checkpoints/weights/step_013400.pt — SHA-256 48e17fc5f41dab63b67436da9dd8ad99be073a9641cb62d3a0e7ccbecff97d8e
• checkpoints/weights/step_020100.pt — SHA-256 c0d96701566cd54120818412b33e3df512c5f5a903a522bafb63051daca9be0c
• checkpoints/weights/step_026800.pt — SHA-256 a9c85260a25f3ca57f79df3165829226a19ad75b5ed1437e48b8f41127b6cf24
• checkpoints/weights/step_033500.pt — SHA-256 ba45fcfd73560241349dda9eb54677a2c6be1a69290cf8dd83a3fd68465342b5
• checkpoints/weights/step_040200.pt — SHA-256 c30b0cc1ac3573bdf40c6dff65a2598d6b4b1cfb39af58398374bfaf6d227ea0

Conditioning

Exact task text (byte-exact match required at deployment — the runtime looks the cached embedding up by SHA-256 of the formatted prompt):

pick a utensil out of a drying rack and place it on a plate

conditioning/text_embedding.pt is the exact cached T5 embedding consumed during training (context length 128, Wan2.2 text stack). The resolved training config sets load_text_encoder: false, so this cached tensor is part of the required inference bundle.

Input processing and normalization

• Two 256x256 RGB cameras (image = agentview, then wrist_image).
• Each camera is converted to a tensor and resized to 224x224.
• Cameras are concatenated horizontally to 224x448.
• Horizon: 33 observations; 32 action transitions at 10 Hz.
• State is 8-D: eef_xyz(3) + eef_rpy(3) + [gripper_width/2, -gripper_width/2] — REAL gripper widths (~0.0002..0.080 m), not binary; inference must feed real widths.
• Action is 7-D: delta XYZ, delta rotation XYZ, and gripper. In this dataset the rotation deltas are identically zero (translation-only teleop).
• Gripper action convention: raw robosuite {-1:open, +1:close} was remapped by the converter to {1:open, 0:close} (LIBERO/RLDS convention).
• Delta/padding mask is [true, true, true, true, true, true, false]; the gripper channel is absolute rather than delta.
• dataset_stats.json contains the exact min/max normalization statistics used by this run (recomputed on this dataset; no pretrained stats).

The exact preprocessing/model implementations are included under training_code/; the resolved config is config.yaml. No license is asserted here for the bundled upstream code; its original terms continue to apply.

Base components

This weights-only checkpoint is not standalone. It references Wan-AI/Wan2.2-TI2V-5B and requires the matching Wan VAE plus the included FastWAM code/configuration. PyTorch .pt files may contain pickled objects; load only in a trusted environment.