pi05-base-p150

Physical Intelligence's pi-0.5 vision-language-action policy (lerobot/pi05_base: SigLIP + Gemma-2B VLM + Gemma-300M action expert, flow matching) running on a single Tenstorrent Blackhole p150a via tt-nn. Two 224x224 camera images + a task prompt (+ normalised robot state, discretised into the prompt) -> a 50-step chunk of 32-dim normalised actions in ~130-165 ms (PCC 0.9921 vs the torch reference; 4/5 LIBERO-spatial tasks closed-loop with the LIBERO fine-tune, matching the CPU reference). Weights are under the Gemma Terms of Use; the port code (Apache-2.0 headers, distributed under the same Gemma terms) is github.com/changh95/tt-pi-0.5 @ f4a1d838, refreshed from the author's tt-metal branch changh95/pi05 of github.com/tenstorrent/tt-metal @ 4c9fbfcceb9 (models/experimental/pi0_5), which the image is also built from.

Runs on p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  changh95/pi05-base-p150 --with-weights
tt-model serve changh95/pi05-base-p150

pull --with-weights downloads the Docker image and the lerobot/pi05_base weights at b211f3d44c36b6acfcf7ae94a64e8e96f75a64ba (into your HF cache; they are not in the image). serve starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

tt-cli

tt serve changh95/pi05-base-p150          # first boot: ~15 GB weights + kernel JIT, then READY
tt model stop changh95/pi05-base-p150

The server listens on the port serve printed (20000, or the next free one). This is not an OpenAI API: GET /v1/models only exists so the ready card does not 404, and tt-model curl does not apply. The real routes are GET /health, GET /info, POST /predict.

Prerequisites

• lerobot/pi05_base is ungated but under the Gemma Terms of Use (accept them).
• The prompt tokenizer google/paligemma-3b-pt-224 is gated: accept the Gemma terms on huggingface.co and hf auth login on the host before serve (the token file under ~/.cache/huggingface is mounted into the container; export HF_TOKEN=... also works). Without it the server refuses to start (PI05_TOKENIZER_REQUIRED=1); set PI05_TOKENIZER_REQUIRED=0 to start anyway and send pre-tokenised tokens.
• One Blackhole p150 (TT_METAL_VISIBLE_DEVICES=0), 20 GB free host RAM during boot (the 14.5 GB fp32 checkpoint is converted to device tensors on every start), and `/.cache/huggingface` with ~15 GB free.

Call it

PORT=20000   # the port `serve` printed
curl -s localhost:$PORT/health
curl -s localhost:$PORT/info | python -m json.tool

# two cameras: [base/exterior, wrist]; state = your robot's proprio, ALREADY normalised
# to [-1, 1] with your dataset's QUANTILES stats (2*(x-q01)/(q99-q01)-1), <= 32 floats
curl -s localhost:$PORT/predict -H 'Content-Type: application/json' -d "{
  \"images\": [\"$(base64 -w0 media/sample_base.png)\", \"$(base64 -w0 media/sample_wrist.png)\"],
  \"prompt\": \"pick up the cube\",
  \"state\": [0.1, -0.2, 0.3, 0.0, 0.0, 0.0, 0.5, -0.5]
}" | python -c 'import json,sys; r=json.load(sys.stdin); print(len(r["actions"]), len(r["actions"][0]), r["normalized"], r["timing_ms"], r["num_tokens"])'

Request fields: images (1-2 base64 PNG/JPEG; a missing second camera is padded with a black image), prompt (task text) or tokens (pre-tokenised PaliGemma ids, <= 224), optional state (<= 32 floats in [-1, 1], default zeros), optional seed (initial flow-matching noise; default: the model's fixed seeded noise -> deterministic policy). Images are squash-resized to 224x224 (no aspect padding), scaled to [-1, 1]. The prompt is built exactly like lerobot: Task: <prompt>, State: b0 ... b31;\nAction: with the state discretised into 256 bins, then right-padded to 224 tokens.

Response: actions (50 x 32 floats, normalised lerobot QUANTILES action space, zero-padded to 32 dims - pi05_base ships no per-feature stats, so denormalise with (a+1)*(q99-q01)/2+q01 from your own dataset and slice to your action dim, e.g. the first 7 for LIBERO), action_horizon 50, action_dim 32, normalized: true, denoising_steps 10, the prompt the server built, num_tokens, prompt_truncated, images_padded, original_sizes, and timing_ms (preprocess, inference, total). Errors: 400 bad input (too many images, undecodable image, > 32 state values, missing prompt/tokens), 503 while warming up, 500 with the exception text.

media/sample_*.png are synthetic placeholders (a red cube on a table) so the command above is copy-pasteable; pi05_base is a base (not task-fine-tuned) checkpoint, so its actions on an arbitrary robot are not directly usable without fine-tuning.

First boot

serve downloads the pinned lerobot/pi05_base snapshot (14.5 GB) into your HF cache, then the container loads it (1-2 min), converts ~3.5 B parameters to bf16/bf8 device tensors (a few minutes), compiles the kernels on the first sample_actions (several minutes cold; cached under `/.cache/tt-model/pi05-base-p150/cacheafterwards) and runs a second warm-up before loggingApplication startup complete. Steady-state inference is ~130-165 ms per 50-step chunk on a p150a. Per-request num_stepsis not supported (the port precomputes per-step conditioning at build time); the server runsPI05_NUM_STEPS` (10). Batch is 1; requests are serialised on the single chip.

Environment switches (baked into the package's serve.env; tt-model serve has no env flag, so change them by re-packaging from tt-model.yaml or by docker run -e KEY=VALUE on the image): PI05_TOKEN_LEN (224; 32 reproduces the README PCC/perf run but cannot carry the state), PI05_NUM_IMAGES (2), PI05_NUM_STEPS (10), PI05_SEED (42), TT_DEVICE_ID (0), PI05_WEIGHTS_DIR / PI05_TOKENIZER_DIR (offline overrides).

Results (from the port's README, single Blackhole p150a)

Metric	Value
PCC vs. torch reference (tests/pcc/test_pcc_pi05_model.py)	0.9921
Latency (README config: 2 images, 32 tokens, 10 steps)	132.7 ms / action chunk
Throughput	376.7 actions/s
LIBERO-spatial closed loop (lerobot/pi05_libero_finetuned_quantiles_v044, chunk 10)	4/5 tasks, same as the CPU reference
LIBERO inference latency p50 (224 tokens)	163.8 ms (CPU torch: 5,357 ms, 33x)
Power (active mean / peak)	69 W / 104 W

Optimisation trajectory (PCC / latency): 0.9977 / 183.4 ms baseline -> bf8 SigLIP + pre-baked adaRMS 169.2 ms -> pre-allocated KV cache 151.4 ms -> KV cache in L1 145.6 ms -> fused rotary_embedding_to_cache 144.6 ms -> precomputed per-step adaRMS 144.1 ms -> precomputed per-(step, layer) modulations 132.7 ms (PCC 0.9921 throughout the last four). The PCC was measured against the port's own torch reference on random inputs; the LIBERO result is the real-input, real-tokenizer validation and used the fine-tuned checkpoint, not pi05_base.

Layout and caveats

• code/models/experimental/pi0_5/ = the fork's package: common/ (configs, weight loader), tt/ (tt-nn model), reference/ (torch reference for PCC), tests/ (pcc / perf / unit / demo, need the tt-metal tree + pytest), server/ (this app + smoke_test.py). code/models/common/lightweightmodule.py is the schema's required tt-metal filler and is not imported.
• The model attends to every prefix token (no padding mask on the VLM, same as the torch reference); image masks are shape-only. A padded (black) second camera is therefore attended, unlike lerobot's masked slot.
• state reaches the model only through the prompt (pi0.5 semantics); the (1, 32) state tensor the API also feeds is ignored by the pi0.5 expert.
• Validated on Blackhole p150a only; Wormhole/p300 untested. Image built with torch 2.11.0+cpu (the fork tree's pin) while the author's host runs used torch 2.7.1+cpu.

Licensing

The upstream model lerobot/pi05_base is licensed gemma - Google's Gemma Terms of Use and Prohibited Use Policy apply, including to downstream distribution. The port code was written by Hyunggi Chang (Tenstorrent, Apache-2.0 headers) and is published under the same terms, since a port cannot grant more than its upstream does. Weights are not redistributed here; they are fetched from the upstream repo under its terms. The tokenizer google/paligemma-3b-pt-224 is gated under the same Gemma terms and is fetched at boot with your token.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	4c9fbfcceb991da0c49b4d4066423aef24f85a66 (dirty tree — the image includes uncommitted changes)
code/ digest	c4329822e57581fa (sha256, first 16 hex digits)
built	2026-09-12T12:06:29+00:00 by tt-model 0.1.0