Aether258/pi05_bi_bread_all3_step10000

pi05_bi — bread (01+02+03 merged), step 10000

openpi pi05_bi checkpoint for a bimanual bread-and-ketchup task with tactile inputs. Step 10000 (1.98 epochs), held-out validation loss 0.0389.

The lowest held-out loss through step 12000 is step 12000 at 0.0379 (Aether258/pi05_bi_bread_all3_step12000); this checkpoint is 0.0003 behind, which is within noise. Published mainly as the second point of the plateau.

This is the three-source bread dataset. An earlier run on sources 01+02 only (775 episodes) is published under Aether258/pi05_bi_bread_all_step*; the two are different datasets and their loss values are not comparable (normalization statistics are computed per dataset).

Task

Single unified instruction for every episode:

Firstly, use the right hand to pick up the bread and put it on the table. Secondly, use the left hand to pick up the ketchup and squeeze it onto the bread.

bread_03 shipped with the placeholder string "perform manipulation task" in its meta/tasks.jsonl -- 587 of the 1,362 episodes, 43% of the merged set. Since prompt_from_task=True feeds this string straight to the model, all three sources were forced onto the instruction above at merge time; otherwise nearly half the data would have trained against an uninformative prompt.

Data

source	episodes	frames	mean frames/ep
KaiyueChen/bread_01	293	196,281	670
KaiyueChen/bread_02	482	244,831	508
KaiyueChen/bread_03	587	276,252	471
merged	1,362	717,364	527

LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet files (total_videos=0). Six camera streams: camera0, camera1, and four tactile sensors (tactile_left_0/1, tactile_right_0/1).

Split

Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:

split	episodes
train	1,226 (644,964 frames)
val_seen (subset of train)	136
val_unseen (held out)	136

Normalization statistics (quantile q01/q99) are computed over the train split only.

Training

config	pi05_bi
hardware	2 x A100-80GB, FSDP
batch size	128
this checkpoint	step 10000 (~1.98 epoch; 1 epoch = 5,039 steps)
planned length	16,000 steps (3.18 epochs)
lr	cosine decay, 1,000 warmup steps: peak 2.5e-5 -> 2.5e-6 over 30,000 steps
	(CosineDecaySchedule defaults -- pi05_bi does not override lr_schedule)
LoRA	rank 16 on the LLM, rank 32 on the action expert
vision tower	fully fine-tuned -- the freeze filter matches only .*llm.*

Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.

step	train	val_seen	val_unseen	gap
0	n/a	0.6770	0.6380	-0.0390
2000	0.0524	0.0681	0.0453	-0.0228
4000	0.0474	0.0533	0.0415	-0.0118
6000	0.0452	0.0507	0.0396	-0.0111
8000	0.0434	0.0468	0.0386	-0.0082
10000	0.0422	0.0468	0.0389	-0.0079
12000	0.0410	0.0431	0.0379	-0.0052

val_unseen fell steadily, paused at step 10000 (0.0386 -> 0.0389), then resumed and set a new low at step 12000 (0.0379). val_seen did the same thing: flat at 0.0468 across steps 8000-10000, then down to 0.0431.

That pause is worth dwelling on, because it looked like a turn and was not. Both validation curves stalling simultaneously while the training loss kept falling is normally a strong overfitting signal -- but with each validation pass covering only ~2,560 frames (about 4.9 episodes per split at ~527 frames each), a +0.0003 move sits inside the sampling noise, and two consecutive points are simply not enough evidence. The same false alarm occurred in the sibling two_tubes_0102 run at steps 6000-8000 and in task2_all at steps 8000-10000; in both cases the next point recovered. Three consecutive points with no trend, plus a monotonically widening gap, is the threshold worth acting on -- two is not.

On the negative gap

The gap is negative throughout, but note it starts at -0.0391 at step 0, before any training. That is the intrinsic difficulty difference between the two sampled episode subsets, not generalization. Its rapid early narrowing (-0.0391 -> -0.0228 -> -0.0118) is the model learning what both subsets have in common, not overfitting. Only the later behaviour -- narrowing again from -0.0112 to -0.0082 while val_unseen slowed -- carries information about overfitting.

Note the training loss is measured on augmented images (random crop to 95%, +-5 deg rotation, colour jitter, applied to all six streams including the four tactile ones) while validation runs with train=False, which skips augmentation. The two columns are therefore not directly comparable.

Contents

checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/bread_all3/
    norm_stats.json            # computed over the train split only