nevertmr/pi05_so101_1task_expert_70ep_260716

π0.5 · SO-101 Cup → Blue Circle — action-expert fine-tune (2026-07-16, 70 eps)

A π0.5 (pi05) vision-language-action policy fine-tuned for an SO-101 arm to pick up a cup and place it on a blue circle. Only the action expert was trained; the PaliGemma vision-language backbone was kept frozen.

This checkpoint is trained on the combination of two datasets — the base demonstrations plus a variation set — for broader coverage than the 50-episode variant.

• Base model: lerobot/pi05_base
• Datasets:
  • nevertmr/so101_cup_bluecircle_1task — 50 episodes / 19,138 frames
  • nevertmr/so101_cup_bluecircle_1task_var — 20 episodes / 6,884 frames
  • Combined: 70 episodes / 26,022 frames
• Framework: LeRobot 0.4.4
• Robot: SO-101, 6-DoF, two cameras (front, wrist)
• Sibling models: pi05_so101_1task_expert_50ep_260716 — same recipe, base 50 episodes only · pi05_so101_1task_full_50ep_260720 — full fine-tune on the base 50 episodes

Task

Single manipulation task — "pick up the cup near the blue circle and place the cup to the blue circle". Observations are two RGB streams (front + wrist, 480×640) plus the 6-DoF joint state; the policy outputs 6-DoF joint action chunks (chunk size 50).

Dataset combination

The two datasets share an identical schema (same features, cameras, 30 fps, so_follower robot, and task string), so they were merged into a single unified LeRobotDataset with LeRobot's official aggregate_datasets utility (episode-level concatenation with unified task/metadata and recomputed statistics):

from lerobot.datasets.aggregate import aggregate_datasets

aggregate_datasets(
    repo_ids=[
        "nevertmr/so101_cup_bluecircle_1task",
        "nevertmr/so101_cup_bluecircle_1task_var",
    ],
    aggr_repo_id="nevertmr/so101_cup_bluecircle_1task_plus_var",
)

Normalization statistics used during training are computed over the merged 70-episode set.

Training

Fine-tuned in action-expert-only mode (train_expert_only=true): the full PaliGemma VLM is frozen and only the action expert and its projections are updated. Same recipe as the 50-episode variant.

Trainable parameters	693 M / 3.62 B total (VLM frozen)
Precision	bfloat16
Batch size	64
Steps	4,000
Optimizer	AdamW, lr 2.5e-5, β=(0.9, 0.95), wd 0.01, grad-clip 1.0
LR schedule	cosine decay with warmup (auto-scaled: warmup 133, decay 4,000)
Gradient checkpointing	on
Seed	1000
Hardware	1× NVIDIA RTX 5090 (32 GB)
Wall-clock	~5 h 49 m (5.22 s/step)
Peak VRAM	27.9 GB

Result: final training loss 0.054 (min 0.050); smooth convergence with a plateau after ~3,000 steps. For reference, the 50-episode run reached 0.046 — the slightly higher loss here is expected, since the added variation episodes make the merged set more diverse, and thus harder to fit, under the same step budget.

Reproduce

# 1) merge the datasets (see snippet above), then:
lerobot-train \
  --job_name=pi05_so101_1task_expert_70ep \
  --dataset.repo_id=nevertmr/so101_cup_bluecircle_1task_plus_var \
  --policy.type=pi05 \
  --policy.pretrained_path=lerobot/pi05_base \
  --policy.train_expert_only=true \
  --policy.compile_model=false \
  --policy.gradient_checkpointing=true \
  --policy.dtype=bfloat16 \
  --policy.device=cuda \
  --policy.push_to_hub=false \
  --steps=4000 \
  --batch_size=64 \
  --save_freq=2000

Note: --job_name and the aggregate dataset id above were modernized to the new naming scheme during the 2026-07-21 repo rename. The original run used --job_name=pi05_cup_to_bluecircle_v1plusvar_expertonly with aggregate id nevertmr/so101_cup_to_bluecircle_v1_plus_var; see train_config.json for the as-run values.

Usage

from lerobot.policies.pi05.modeling_pi05 import PI05Policy

policy = PI05Policy.from_pretrained("nevertmr/pi05_so101_1task_expert_70ep_260716")
policy.eval().to("cuda")

# Feed an observation through the saved preprocessor, then call select_action.
# observation = {"observation.state": ..., "observation.images.front": ...,
#                "observation.images.wrist": ...,
#                "task": "pick up the cup near the blue circle and place the cup to the blue circle"}
# action = policy.select_action(preprocessor(observation))

The repository ships the matching pre/post-processors (normalization from the merged dataset, tokenizer, device steps); load them with make_pre_post_processors(policy_cfg=policy.config, pretrained_path=...) for inference.

Note: the saved processor config references the relative_actions_processor / absolute_actions_processor steps (serialized with enabled: false, i.e. no-ops). LeRobot 0.4.4 does not register these steps, so on that exact version register them manually (a faithful port of lerobot/processor/relative_action_processor.py from a newer release) before loading the processors; newer LeRobot versions load them out of the box.

Limitations

• Trained on a single task with 70 demonstrations; expect it to generalize only to that cup-to-blue-circle setup and the camera placement used for collection.
• Validated offline (checkpoint loads and produces finite 6-DoF actions on held dataset samples). On-robot success rate was not measured in this release.
• Since the VLM is frozen, language grounding comes entirely from the base model.

License

The base model lerobot/pi05_base is released under the Gemma license (it contains PaliGemma / Gemma components). This fine-tune keeps those weights (frozen VLM) and therefore inherits the same terms.