vrfai/pi05-libero-gguf

π0.5 (pi05) - LIBERO base (GGUF for vla.cpp)

GGUF conversion of lerobot/pi05_libero_base for inference with vla.cpp, a lightweight C++ inference engine for Vision-Language-Action models built on top of llama.cpp.

π0.5 pairs a PaliGemma vision-language backbone (gemma_2b variant) with a Gemma-300m action expert running a flow-matching head. It differs from π0 in three ways that the vla.cpp runtime reproduces exactly: /

• State-in-prompt - the proprioceptive state is per-dimension quantile-normalised, discretised into 256 bins, and appended to the text prompt (Task: <instruction>, State: <bins>;\nAction: ) instead of being fed as a state token.
• adaRMS action expert - each expert block uses adaptive RMS norm: a dense(cond) projection of the flow-matching timestep embedding produces per-layer scale/shift/gate, applied to gated residuals.
• Quantile action normalisation - actions are un-normalised with q01/q99 quantiles (baked into the GGUF) rather than mean/std.

Usage

# Terminal 1 - serve (use the CUDA build for inference)
./build-cuda/vla-server --bind tcp://*:5555 \
    pi05-libero.gguf

# Terminal 2 - drive a LIBERO episode (inside the LIBERO uv venv)
python eval/client/run_sim_client_direct.py \
    --arch pi05 \
    --task libero_object --task-id 0 --n-episodes 10 \
    --n-action-steps 10 \
    --max-length 200 \
    --stats-json <LIBERO meta/stats.json> \
    --vla-addr tcp://localhost:5555

Notes:

• π0.5 uses the gated google/paligemma-3b-pt-224 tokenizer. Run huggingface-cli login and accept the licence, or point --tokenizer at a local copy.
• --stats-json must point at a LIBERO meta/stats.json that carries q01/q99 for observation.state (π0.5 discretises the state with quantiles). If omitted, the client auto-downloads lerobot/libero meta/stats.json.
• --n-action-steps 10 matches the checkpoint's n_action_steps (chunk size 50, 10 Euler denoise steps). --max-length 200 matches the training prompt budget.

Validation

Runtime parity (vs the lerobot PI05Pytorch.sample_actions reference, F32 both sides, image embeddings fed in to isolate the action stack): worst |Δ| = 1.98e-4 over the full prefill + 10-step Euler denoise - well within the 2e-3 threshold.

LIBERO smoke test (libero_object / task 0, 10 episodes, RTX 3090):

Hardware	n_act	Success	client/step
RTX 3090 (sm_86)	10	80.0% (8/10)	69.7 ms

This is a single-task smoke run to confirm the end-to-end pipeline, not the full libero_object sweep (10 tasks × N episodes).

License

Weights follow the upstream license of lerobot/pi05_libero_base (Apache-2.0). The vla.cpp conversion tooling and inference engine are MIT-licensed.