emodel30

EBT (energy-based transformer, CLIP ViT-B/16 + AdaLN energy transformer) policy for a real Franka Panda two-object insertion task: pick a yellow slotted cylinder and insert it into the round hole of a white board, then insert a green stepped block into the stepped square hole, then return home.

• Training data: 73 SpaceMouse teleop demos (69 train / 4 val episodes, 101,675 train frames at 20 Hz), LeRobot v3 dataset jalal06/real-franka-insertion (not on the Hub yet).
• Checkpoint: step 34,662 of run ebt_real73_b88_h18e16, about 30 passes over the training frames (batch 88).
• Inputs: observation.state = 7 joint positions (rad) + gripper width (m); observation.images.main and observation.images.wrist (400x224 RGB, resized to 224x224 inside the CLIP encoder); 2 observation frames.
• Outputs: chunks of 18 actions, 16 executed per chunk; each action = 7 absolute joint-position targets (rad) + gripper command (-1 open, +1 close), for a joint-impedance controller at 20 Hz.
• Normalization: saved LeRobot pre/post-processors (min-max stats from the train episodes).

Loading needs LeRobot 0.5.2 and the lerobot_policy_ebt plugin (registers the ebt and dit_clip policy types). config.json points clip_name at a local copy of openai/clip-vit-base-patch16; set it to openai/clip-vit-base-patch16 (or your local path) before loading on another machine.