Aether258/pi05_bi_two_tubes_0102_step6000

pi05_bi — two_tubes (01+02 merged), step 6000

openpi pi05_bi checkpoint from a bimanual two-tube pick-and-place run with tactile inputs. This is the step-6000 checkpoint, which had the lowest held-out validation loss of the run up to step 8000.

Task

Single unified instruction for every episode:

Firstly, use the left hand to pick up the blue tube, and then use the right hand to pick up the green tube. Next, use the left hand to place the blue tube back firstly, and then use the right hand to place the green tube back.

two_tubes_02 shipped with the placeholder string "perform manipulation task" in its meta/tasks.jsonl. Since prompt_from_task=True feeds this string straight to the model, the two sources were forced onto the single instruction above at merge time -- otherwise the model would be taught that two different instructions mean the same motion.

Data

source	episodes	frames
KaiyueChen/two_tubes_01	519	425,115
KaiyueChen/two_tubes_02	500	377,604
merged	1,019	802,719

LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet files (total_videos=0). Six camera streams: camera0, camera1, and four tactile sensors (tactile_left_0/1, tactile_right_0/1).

Split

Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:

split	episodes
train	917
val_seen (subset of train)	102
val_unseen (held out)	102

Normalization statistics (quantile q01/q99) are computed over the train split only.

Training

config	pi05_bi
hardware	2 x A100-80GB, FSDP
batch size	128
this checkpoint	step 6000 (~1.06 epoch; 1 epoch = 5,639 steps)
planned length	20,000 steps
lr	cosine decay, 1,000 warmup steps: peak 2.5e-5 -> 2.5e-6 over 30,000 steps
	(CosineDecaySchedule defaults -- pi05_bi does not override lr_schedule; the peak_lr=2e-4 / decay_steps=100000 block in config.py is referenced only by pi05_single*)
LoRA	rank 16 on the LLM, rank 32 on the action expert
vision tower	fully fine-tuned -- the freeze filter matches only .*llm.*

Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.

step	train	val_seen	val_unseen	gap
0	0.5525	0.4968	0.5261	0.0293
2000	0.0504	0.0504	0.0608	0.0104
4000	0.0475	0.0467	0.0576	0.0109
6000	0.0448	0.0437	0.0543	0.0105
8000	0.0439	0.0423	0.0550	0.0127

val_unseen fell monotonically through step 6000, then ticked up by 0.0007 at step 8000 while val_seen kept improving and the gap widened. Each validation pass covers only ~2,560 frames (roughly 3-4 episodes of ~780 frames), so a change of that size is within noise -- whether step 6000 is the true optimum is not yet settled.

Contents

checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/two_tubes_0102/
    norm_stats.json            # computed over the train split only