mug_model10

DiT (CLIP ViT-B/16 + AdaLN-Zero diffusion transformer, DDPM 100 steps, epsilon prediction) policy for a real Franka Panda task: "Hang the mug on the holder." Vision only (no proprioception).

• Training data: 100 SpaceMouse teleop demos (96 train / 4 val episodes, 22,746 train frames at 20 Hz), LeRobot v3 dataset jalal06/real-franka-mug-hang (not on the Hub).
• Checkpoint: step 2,585 of run dit_mug100_vis_b88_h18e16, about 10 passes over the training frames (batch 88).
• Inputs: observation.images.main and observation.images.wrist (400x224 RGB in [0,1], resized to 224x224 inside the CLIP encoder), 2 observation frames. observation.state (15-D) must still be supplied but is zeroed inside the model (use_state: false).
• Outputs: chunks of 18 actions, 16 executed per chunk. Each action is [dx, dy, dz (m), rx, ry, rz (rotation vector, rad), gripper]: the offset from the measured hand pose to the target, p_target = p + d, R_target = Exp(r) @ R (base frame), executed by deoxys OSC_POSE. Clip to 40 mm / 0.25 rad as in the demos. Gripper: 0 = closed, 1 = open.
• Compose every action with the hand pose measured at the step it is sent, not the pose at the start of the chunk.
• The demonstrations end while still holding the mug (no release), so the policy will not let go.

Loading needs LeRobot 0.5.2 and the lerobot_policy_ebt plugin with the use_state option. config.json points clip_name at a cluster path; set it to openai/clip-vit-base-patch16 before loading on another machine.