YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

pi0.5 Inference Server

pi05-server runs the pi0.5 vision-language-action model on the board's NPU and serves it over HTTP. Your robot sends camera images, its current state and a task in plain language. The server sends back a chunk of 50 future actions.

The model runs with W8A8 quantization: 8-bit weights and 8-bit activations for the language model and the action expert. This is the default and the recommended configuration.

How it works

   YOUR SIDE (robot / PC)                          THE BOARD (aarch64 + NPU)
   ──────────────────────────────                  ─────────────────────────────────────────
                                                   $ ./pi05-server ... --port 8080
                                                     At startup (once):
                                                       load the W8A8 weights
                                                       deploy the model on the NPU
                                                     Then the server waits for requests.

   3 camera frames (JPEG/PNG)┐
   robot state             ──┼── POST /v1/actions ──►  1. decode the images
   task ("pick up the cup") β”€β”˜      (JSON)             2. vision encoder         (NPU)
                                                       3. language model prefill (NPU, W8A8)
                                                       4. action expert,
                                                          10 denoising steps     (NPU, W8A8)
                                                                β”‚
   ◄─────────────── 50 x 32 actions (JSON) β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

   unnormalize the actions,
   execute them on the robot,
   send the next observation ...

The model is loaded and deployed once, when the server starts. A request pays only for the inference itself (vision encoding, prefill and action sampling), so the server stays running between requests.

The NPU runs one inference at a time. When several requests arrive together, they are queued and served in order.

Files

File Description
pi05-server the server executable
model.safetensors pi0.5 checkpoint (vision encoder and embeddings)
pi05_llm_w8a8.gguf W8A8 language model
pi05_ae_w8a8.gguf W8A8 action expert

model.safetensors is the G1-finetuned pi0.5 checkpoint niravpanchalmerai/g1_pi05_v6:

hf download niravpanchalmerai/g1_pi05_v6 model.safetensors --local-dir g1_pi05_v6

Starting the server

On the board:

sh pi05-server-0.1.0-Linux.sh --skip-license --exclude-subdir

./pi05-server -w model.safetensors \
              -w-llm-w8a8 pi05_llm_w8a8.gguf \
              -w-ae-w8a8  pi05_ae_w8a8.gguf \
              --host 0.0.0.0 --port 8080
Option Description Default
-w FILE pi0.5 model.safetensors required
-w-llm-w8a8 FILE W8A8 language model pi05_llm_w8a8.gguf
-w-ae-w8a8 FILE W8A8 action expert pi05_ae_w8a8.gguf
--host ADDR address to listen on 0.0.0.0
--port N port to listen on 8080
--npu-idx N NPU cluster, 0 or 1 0

The server is ready when it prints that it is listening. You can also poll GET /health.

API

POST /v1/actions: get the next action chunk

Request (Content-Type: application/json)

{
  "prompt": "pick up the red cup and place it in the bowl",
  "state":  [0.12, -0.40, 0.05, 0.88, -0.10, 0.33, 1.00],
  "images": [
    "/9j/4AAQSkZJRgABAQAAAQABAAD...",
    "/9j/4AAQSkZJRgABAQAAAQABAAD...",
    "/9j/4AAQSkZJRgABAQAAAQABAAD..."
  ],
  "seed": 0
}
Field Type Required Description
prompt string yes The task in plain language.
state array of numbers no Current robot state, normalized to [-1, 1], up to 32 values. Missing values are treated as 0.
images array of strings yes Exactly 3 camera images, base64-encoded JPEG or PNG. Images of any size are accepted and resized to 224Γ—224 on the board.
seed integer no Seed of the initial noise for the action sampler. Leave it out for a fresh random seed on each request. Set it for reproducible results.
noise array of 50 arrays of 32 numbers no The initial noise itself, drawn from N(0, 1), shape [50, 32]. Use it only for validation (see below). When given, seed is ignored.

Camera order matters. Send the 3 cameras in the same order that was used when the model was trained (for example: base camera, left wrist, right wrist).

Validating against a reference (optional). You do not need noise in normal operation. Both seed and noise are optional, and requests without them work as usual. seed makes the server's results reproducible, but the same seed does not give the same noise as NumPy or PyTorch, because the random number generators differ. To compare the server's actions with your own reference run of pi0.5 (for example a PyTorch/openpi run in fp32), generate the noise once, pass it to your reference as the sampler's initial noise and send the same values in noise. Both sides then start from identical noise, so any remaining difference comes from the model execution (W8A8 on the NPU).

noise = np.random.default_rng(0).standard_normal((50, 32)).astype(np.float32)
# reference: your pi0.5 sampler, starting from `noise`
# server:    requests.post(..., json={..., "noise": noise.tolist()})

Response (200 OK)

{
  "actions": [
    [ 0.0213, -0.1180,  0.0457,  0.3021, -0.0042,  0.1175,  0.9810, 0.0, ... ],
    [ 0.0251, -0.1123,  0.0490,  0.2987, -0.0051,  0.1150,  0.9802, 0.0, ... ],
    ...
  ],
  "shape": [50, 32],
  "seed": 0,
  "timing": {
    "encode_s":   0.000,
    "prefill_s":  0.000,
    "sample_s":   0.000,
    "total_s":    0.000
  }
}
Field Description
actions 50 rows (time steps) Γ— 32 values, normalized.
shape always [50, 32]
seed the seed the initial noise came from: yours, or the random one the server picked. Send it back as seed to reproduce the result. Not present when you sent noise.
timing time spent on the NPU for this request, in seconds

Using the actions

  • The model always outputs 32 values per step. Use only the first N, where N is your robot's action dimension (for example 7 for a 6-DoF arm with a gripper). Ignore the rest.
  • The actions are normalized. Unnormalize them with the same statistics that were used to normalize the state and actions in training.
  • A common pattern is to execute the first part of the chunk, then send a new observation, rather than executing all 50 steps open-loop.

Errors

Invalid requests get 400 Bad Request with a JSON body:

{ "error": { "message": "images: 2 given, 3 required", "type": "invalid_request_error" } }

Typical causes: missing prompt or images, not exactly 3 images, an image that cannot be decoded, more than 32 state values, or noise that is not 50 Γ— 32 numbers. A failure on the board itself returns 500.

GET /health

{ "status": "ok" }

GET /v1/info

{
  "model": "pi0.5",
  "precision": "w8a8",
  "images": 3,
  "image_size": [224, 224],
  "state_dim": 32,
  "action_horizon": 50,
  "action_dim": 32,
  "denoising_steps": 10
}

Examples

Test from the PC (through an ssh tunnel to the board):

ssh -J <user>@<labhost> -N -L 8080:localhost:8080 root@10.0.0.2
curl -s localhost:8080/health
curl -s localhost:8080/v1/info

python3 tools/pi05_client.py --server http://localhost:8080 \
  test_data/pi05-inputs/cam0_cam_left_high.png test_data/pi05-inputs/cam1_cam_left_wrist.png \
  test_data/pi05-inputs/cam2_cam_right_wrist.png \
  --prompt-file test_data/pi05-inputs/prompt.txt \
  --state-file  test_data/pi05-inputs/state.txt \
  --noise       test_data/pi05-inputs/noise.f32

--noise sends the initial noise from a raw f32 [50][32] file (the format of pi05-runner --noise), so the server's actions can be compared with the runner's or a reference run from the same noise. Leave it out and pass --seed N for a seeded random noise instead.

Downloads last month
19
GGUF
Model size
0.3B params
Architecture
pi05_ae_w8a8
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support