nevertmr/pi05_so101_1task_full_50ep_260720

π0.5 · SO-101 Cup → Blue Circle — full fine-tune (2026-07-20, 50 eps)

A π0.5 (pi05) vision-language-action policy fine-tuned for an SO-101 arm to pick up a cup and place it on a blue circle. Unlike the expert-only sibling, the entire model was trained — PaliGemma VLM (vision encoder + language model) and the action expert together (train_expert_only=false, freeze_vision_encoder=false).

• Base model: lerobot/pi05_base (revision a538eb27, the last pre-relative-actions revision, matching LeRobot 0.4.4)
• Dataset: nevertmr/so101_cup_bluecircle_1task (50 episodes · 19,138 frames · 30 fps)
• ▶ Data collection video (YouTube): https://youtu.be/s7c5Ls-Nkd0
• Framework: LeRobot 0.4.4
• Robot: SO-101, 6-DoF, two cameras (front, wrist)
• Sibling models: 260716_50ep (expert-only) · 260716_70ep (expert-only, +variation set)

Task

Single manipulation task — "pick up the cup near the blue circle and place the cup to the blue circle". Observations are two RGB streams (front + wrist, 480×640) plus the 6-DoF joint state; the policy outputs 6-DoF joint action chunks (chunk size 50).

Training

Full fine-tuning (all 3.62 B parameters trainable — no frozen modules), on a single NVIDIA DGX Spark (GB10, 128 GB unified memory, aarch64).

Trainable parameters	3.62 B / 3.62 B (nothing frozen)
Precision	bfloat16
Batch size	32 (recipe: bs64 exceeded the 25 s/step budget on GB10 → halved per protocol)
Steps	8,000 (doubled from 4,000 to keep total samples = 256 K, identical to the bs64×4k reference)
Optimizer	AdamW, lr 2.5e-5, β=(0.9, 0.95), wd 0.01, grad-clip 1.0
LR schedule	cosine decay with warmup (auto-scaled: warmup 266, decay 8,000)
Gradient checkpointing	on
Seed	1000
Hardware	1× NVIDIA DGX Spark (GB10, 128 GB unified, aarch64)
Wall-clock	~30 h 14 m (mean 13.5 s/step, range 12.9–14.3)
Peak memory	~75.6 GB of 128 GB unified (host-RAM-derived; no dedicated VRAM counter on GB10)

Result: final training loss 0.017 (min 0.017, first reached at step 7,450; 0.195 at step 50); mean bucket loss was still falling ~8 % per 1,000 steps through 6–7 k but slowed to ~3 % over the final interval — improvement continuing but tapering, with no full plateau within the 8 k budget.

Loss curve

steps	0–1k	1–2k	2–3k	3–4k	4–5k	5–6k	6–7k	7–8k
mean loss	0.0730	0.0386	0.0300	0.0247	0.0222	0.0205	0.0188	0.0182

Reproduce

lerobot-train \
  --job_name=pi05_cup_to_bluecircle_v1 \
  --dataset.repo_id=nevertmr/so101_cup_bluecircle_1task \
  --policy.type=pi05 \
  --policy.pretrained_path=lerobot/pi05_base \
  --policy.compile_model=false \
  --policy.gradient_checkpointing=true \
  --policy.dtype=bfloat16 \
  --policy.device=cuda \
  --policy.push_to_hub=false \
  --steps=8000 \
  --batch_size=32 \
  --save_freq=1000

Note: the dataset id above was modernized to the new naming scheme during the 2026-07-21 repo rename. The original run used --dataset.repo_id=nevertmr/so101_cup_to_bluecircle_v1 and --save_freq=250; see train_config.json for the as-run values.

Notes for LeRobot 0.4.4 on aarch64 (DGX Spark):

• Install the pinned transformers fork (pi extra is absent from the PyPI wheel): pip install "transformers @ git+https://github.com/huggingface/transformers.git@fix/lerobot_openpi" "scipy>=1.10.1,<1.15"
• Use lerobot/pi05_base revision a538eb273274eb30f126a118f39dbc0ee212c883 — the current head references relative_actions_processor, which 0.4.4 does not register.
• Default PyPI torch on linux-aarch64 is CPU-only; install torch==2.10.0 torchvision==0.25.0 from https://download.pytorch.org/whl/cu130.

Usage

from lerobot.policies.pi05.modeling_pi05 import PI05Policy

policy = PI05Policy.from_pretrained("nevertmr/pi05_so101_1task_full_50ep_260720")
policy.eval().to("cuda")

# Feed an observation through the saved preprocessor, then call select_action.
# observation = {"observation.state": ..., "observation.images.front": ...,
#                "observation.images.wrist": ...,
#                "task": "pick up the cup near the blue circle and place the cup to the blue circle"}
# action = policy.select_action(preprocessor(observation))

The repository ships the matching pre/post-processors (normalization from the fine-tuning dataset, tokenizer, device steps); load them with make_pre_post_processors(policy_cfg=policy.config, pretrained_path=...) for inference. Saved with LeRobot 0.4.4 — the processor configs contain only steps that 0.4.4 registers (no manual step registration needed).

Limitations

• Trained on a single task with 50 demonstrations; expect it to generalize only to that cup-to-blue-circle setup and the camera placement used for collection.
• Validated offline (checkpoint loads and produces finite 6-DoF action chunks on held dataset samples). On-robot success rate was not measured in this release.
• Full fine-tuning on 50 episodes can drift the VLM's general language grounding toward this single instruction; for multi-task use, prefer the base model or lighter tuning.

License

The base model lerobot/pi05_base is released under the Gemma license (it contains PaliGemma / Gemma components). This fine-tune updates and redistributes those weights and therefore inherits the same terms.