EndeavoringYoon/VINE2_sim_sft_pi05

VINE2 sim SFT policy (pi05_base → π⁰)

Supervised finetune of lerobot/pi05_base on 300 simulated demonstrations of "Pick orange box and place it on white box" (ROBOTIS FFW-SG2, left arm).

This is the initial policy π⁰ for a RECAP / π*0.6 style improvement loop: it is rolled out in simulation while a human takes over to correct mistakes, and the resulting corrections train the advantage-conditioned policy.

Numbers

Rolled out in the calibrated simulator, 12 episodes, initial conditions drawn from the training distribution:

	success	box lifted	median distance to target
pi05_base, zero shot	0 %	0 %	471 mm
SFT on 100 demos	8 %	42 %	263 mm
SFT on 300 demos (this)	25 %	92 %	49 mm

Grasping is essentially solved; the remaining failures are in placement — the box is carried to the target and then toppled or set down off-centre.

Training

base	lerobot/pi05_base
data	300 sim demonstrations, 39,554 frames, 2 cameras
epoch	5 of 12 — chosen by lowest validation loss, not by best.pt
chunk	chunk_size=10, n_action_steps=10 (1 s at 10 Hz)
precision	bfloat16, vision encoder frozen, 3 backbone layers unfrozen
lr / batch	1e-4 / 8

chunk_size matters: the default of 50 is one second at the paper's 50 Hz, but this data is 10 Hz, where 50 steps is a five second open loop.

Do not use the trainer's best.pt for a BC run — it selects on advantage conditioning accuracy, which is at chance when conditioning is disabled.

Observations

key	shape
observation.images.cam_head	3 × 376 × 672
observation.images.cam_wrist_left	3 × 240 × 424
observation.state	8 (7 arm joints + gripper)
action	8

cam_wrist_right from the real robot is omitted: the right arm is not actuated in this task, so the channel is a near-constant image and cannot be calibrated in simulation.

Loading

from safetensors.torch import load_file
state = load_file("model.safetensors")
policy.load_state_dict(state, strict=False)

Weights are bf16 throughout, matching how the policy runs.