Aether258/pi05_bi_task2_all_step8000

pi05_bi — task2 (dish washing), step 8000

openpi pi05_bi checkpoint for a bimanual dish-and-sponge task with tactile inputs. Step 8000 (1.24 epochs), held-out validation loss 0.0535 (best through step 12000 is step 12000 at 0.0534).

Task

Single unified instruction for every episode:

Use the left hand to pick up the dish, and then use the right hand to pick up the sponge to brush the dish. Finally, put all things back.

Only task2_01 shipped this string; task2_02 and task2_03 both carried the placeholder "perform manipulation task" in their meta/tasks.jsonl. Since prompt_from_task=True feeds that string straight to the model, all three sources were forced onto the instruction above at merge time -- otherwise 76% of the episodes (723 of 948) would have trained against an uninformative prompt. A trailing space in the original string was stripped.

Data

source	episodes	frames
KaiyueChen/task2_01	225	222,531
KaiyueChen/task2_02	298	291,661
KaiyueChen/task2_03	425	402,110
merged	948	916,302

LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet files (total_videos=0). Six camera streams: camera0, camera1, and four tactile sensors (tactile_left_0/1, tactile_right_0/1). Mean episode length ~966 frames.

Split

Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:

split	episodes	held out from
train	854
val_seen (subset of train)	94
val_unseen (held out)	94	22 / 30 / 42 from sources 01 / 02 / 03

Normalization statistics (quantile q01/q99) are computed over the train split only.

Training

config	pi05_bi
hardware	2 x A100-80GB, FSDP
batch size	128
this checkpoint	step 8000 (~1.24 epoch; 1 epoch = 6,444 steps)
planned length	20,000 steps
lr	cosine decay, 1,000 warmup steps: peak 2.5e-5 -> 2.5e-6 over 30,000 steps
	(CosineDecaySchedule defaults -- pi05_bi does not override lr_schedule; the peak_lr=2e-4 / decay_steps=100000 block in config.py is referenced only by pi05_single*)
LoRA	rank 16 on the LLM, rank 32 on the action expert
vision tower	fully fine-tuned -- the freeze filter matches only .*llm.*

Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.

step	train	val_seen	val_unseen	gap
0	0.6399	0.6621	0.6508	-0.0113
2000	0.0535	0.0671	0.0662	-0.0009
4000	0.0470	0.0574	0.0573	-0.0001
6000	0.0448	0.0565	0.0566	+0.0001
8000	0.0430	0.0513	0.0535	+0.0022
10000	0.0420	0.0514	0.0552	+0.0038
12000	0.0421	0.0493	0.0534	+0.0041

val_unseen fell steeply through step 8000 (0.0535), read 0.0552 at step 10000, then returned to 0.0534 at step 12000 -- the step-10000 reading was noise, not a turn. Improvement has nonetheless nearly stopped: the 4,000 steps from 8000 to 12000 bought 0.0001. Over that same span val_seen improved 4% (0.0513 -> 0.0493) and the gap widened from 0.0022 to 0.0041.

The gap is still small in absolute terms. The sibling two_tubes_0102 run had a gap of 0.0135 at its step 12000 -- more than 3x larger -- and its val_unseen had been flat for 6,000 steps by then. task2 is overfitting far less at the same step count, which is why training continued past this checkpoint.

The early-step gap is slightly negative (-0.0009 at step 2000). That is not evidence of good generalization: each validation pass covers only ~2,560 frames, which at ~966 frames per episode is about 2-3 episodes per split, so the sign of the gap early on is dominated by which particular episodes landed in each split. Only the val_unseen absolute trend is reliable, since the same episodes and the same rng are used at every evaluation.

Note that the training loss is measured on augmented images (random crop to 95%, +-5 deg rotation, colour jitter, applied to all six streams including the four tactile ones) while validation runs with train=False, which skips augmentation. The two numbers are therefore not directly comparable; on this run val_seen still sits above the training loss at every step, unlike the two_tubes_0102 run where it dropped below after ~2.5 epochs of memorization.

Contents

checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/task2_all/
    norm_stats.json            # computed over the train split only