Aether258/pi05_bi_two_tubes_0102_step16000

pi05_bi — two_tubes (01+02 merged), step 16000

openpi pi05_bi checkpoint from a bimanual two-tube pick-and-place run with tactile inputs. Step 16000 (2.84 epochs), held-out validation loss 0.0551 (best so far is step 14000 at 0.0537).

Task

Single unified instruction for every episode:

Firstly, use the left hand to pick up the blue tube, and then use the right hand to pick up the green tube. Next, use the left hand to place the blue tube back firstly, and then use the right hand to place the green tube back.

two_tubes_02 shipped with the placeholder string "perform manipulation task" in its meta/tasks.jsonl. Since prompt_from_task=True feeds this string straight to the model, the two sources were forced onto the single instruction above at merge time -- otherwise the model would be taught that two different instructions mean the same motion.

Data

source	episodes	frames
KaiyueChen/two_tubes_01	519	425,115
KaiyueChen/two_tubes_02	500	377,604
merged	1,019	802,719

LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet files (total_videos=0). Six camera streams: camera0, camera1, and four tactile sensors (tactile_left_0/1, tactile_right_0/1).

Split

Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:

split	episodes
train	917
val_seen (subset of train)	102
val_unseen (held out)	102

Normalization statistics (quantile q01/q99) are computed over the train split only.

Training

config	pi05_bi
hardware	2 x A100-80GB, FSDP
batch size	128
this checkpoint	step 16000 (~2.84 epoch; 1 epoch = 5,639 steps)
planned length	20,000 steps
lr	peak 2e-4, 1,000 warmup steps, cosine decay over 100,000
LoRA	rank 16 on the LLM, rank 32 on the action expert
vision tower	fully fine-tuned -- the freeze filter matches only .*llm.*

Steps 0-10000 ran on one host; the run was then resumed from the step-10000 checkpoint on a fresh host with --resume. --resume restores model weights and optimizer state but not the data loader position, so the batch sequence after step 10000 differs from what an uninterrupted run would have seen. This is why step 12000 here reads 0.0538 while the original run's step 12000 read 0.0542 -- the weights genuinely differ, it is not measurement noise.

Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.

step	train	val_seen	val_unseen	gap
0	0.5525	0.4968	0.5261	0.0293	初始 run
2000	0.0553	0.0504	0.0608	0.0104	初始 run
4000	0.0490	0.0467	0.0576	0.0109	初始 run
6000	0.0460	0.0437	0.0543	0.0106	初始 run
8000	0.0441	0.0423	0.0550	0.0127	初始 run
10000	0.0435	0.0416	0.0542	0.0126	初始 run / 恢复点
12000	0.0420	0.0403	0.0538	0.0135	续训
14000	0.0404	0.0387	0.0537	0.0150	续训
16000	0.0395	0.0383	0.0551	0.0168	续训 / 训练在此停止

val_unseen fell monotonically through step 6000 and then stopped: it oscillated in 0.0537-0.0551 for the next 10,000 steps with no trend (0.0543 -> 0.0550 -> 0.0542 -> 0.0538 -> 0.0537 -> 0.0551). Over that same span val_seen improved 12.4% (0.0437 -> 0.0383) and the gap widened 60% (0.0105 -> 0.0168), monotonically at every point. Training was stopped at step 16000 rather than the planned 20,000 on that basis.

The reliable signal is not the final uptick -- 0.0550 also appeared at step 8000 and then fell back -- but the absence of any trend across six consecutive points while the gap grew monotonically. Capacity gained after step 6000 went into fitting the training episodes rather than transferring.

Note that val_seen (0.0383) is now below the training loss (0.0395). This is expected, not an anomaly: the training loss is measured on augmented images (random crop to 95%, +-5 deg rotation, colour jitter, applied to all six streams including the four tactile ones) while validation runs with train=False, which skips augmentation. The augmentation therefore costs 0.0012 of loss, while the generalization gap is 0.0168 -- 14x larger. The augmentation is not buying generalization at this strength.

Each validation pass covers only ~2,560 frames (roughly 3-4 episodes of ~780 frames), so single-point moves under +-0.001 are within noise.

Contents

checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/two_tubes_0102/
    norm_stats.json            # computed over the train split only