SleepMastger/dish-utensil-flashwam-v2

Dish-utensil FlashWAM v2

Public bundle for the dish-utensil manipulation model trained by George on the SleepMastger/dish-utensil-manipulation dataset (100 teleoperated Franka episodes: 50 fork + 50 spoon, 10 Hz, 42,863 frames).

This is the v2 relabel: the same 100 episodes, retrained with a new task string (below) that replaced the dataset's original instruction in meta/tasks.jsonl. Only the label and its cached T5 embedding differ from the v1 runs; frames, actions, and normalization are identical.

Status

This is the complete run: all 30 epochs (40,200 steps), checkpointed every 5 epochs. The model has not yet been evaluated on a robot.

Architecture: FlashWAM M1 fused-KV, fixed-RoPE, 1-layer action expert. Trained FROM SCRATCH (no LIBERO resume): Wan2.2 base video expert + ActionDiT linear-interp init, 30 epochs, global batch 32, cosine LR 1e-4, bf16 DeepSpeed ZeRO-1 on 4x H200.

Checkpoints follow the same layout as the existing WAM repositories:

• checkpoints/weights/step_006700.pt — SHA-256 2eaf8b8e3836ec6fd44044e23118a63f691ded0db7470d46bc525e9d3251ee33
• checkpoints/weights/step_013400.pt — SHA-256 e1a0873b2e95484e80637c0871c6e6c723452989aea3409320cc1bd9b7ace164
• checkpoints/weights/step_020100.pt — SHA-256 a5aecfe80029badab7ddb6607d7e205315d30b7af250a200b6351a3ff09a5c05
• checkpoints/weights/step_026800.pt — SHA-256 1e60d92d6e92ea92c994eb3ddea794d6485ac25527925e3f2511dac43823c3f2
• checkpoints/weights/step_033500.pt — SHA-256 496a578a21fd8725d41f8d85ae18dcc687d93cfb552fab99ce8169aab62cfa20
• checkpoints/weights/step_040200.pt — SHA-256 9212adda8ef0c361f3468e17eebc99d1607d940329195fa332b5806f761877d6

Conditioning

Exact task text (byte-exact match required at deployment — the runtime looks the cached embedding up by SHA-256 of the formatted prompt):

pick the dishes out of the rack and place them on the table, then pick and place the spoon in the gray dish and fork in the pink dish

conditioning/text_embedding.pt is the exact cached T5 embedding consumed during training (context length 128, Wan2.2 text stack). The resolved training config sets load_text_encoder: false, so this cached tensor is part of the required inference bundle.

Input processing and normalization

• Two 256x256 RGB cameras (image = agentview, then wrist_image).
• Each camera is converted to a tensor and resized to 224x224.
• Cameras are concatenated horizontally to 224x448.
• Horizon: 33 observations; 32 action transitions at 10 Hz.
• State is 8-D: eef_xyz(3) + eef_rpy(3) + [gripper_width/2, -gripper_width/2] — REAL gripper widths (~0.0002..0.080 m), not binary; inference must feed real widths.
• Action is 7-D: delta XYZ, delta rotation XYZ, and gripper. In this dataset the rotation deltas are identically zero (translation-only teleop).
• Gripper action convention: raw robosuite {-1:open, +1:close} was remapped by the converter to {1:open, 0:close} (LIBERO/RLDS convention).
• Delta/padding mask is [true, true, true, true, true, true, false]; the gripper channel is absolute rather than delta.
• dataset_stats.json contains the exact min/max normalization statistics used by this run (recomputed on this dataset; no pretrained stats).

The exact preprocessing/model implementations are included under training_code/; the resolved config is config.yaml. No license is asserted here for the bundled upstream code; its original terms continue to apply.

Base components

This weights-only checkpoint is not standalone. It references Wan-AI/Wan2.2-TI2V-5B and requires the matching Wan VAE plus the included FastWAM code/configuration. PyTorch .pt files may contain pickled objects; load only in a trusted environment.