xvla-base-p150

X-VLA-base (the cross-embodiment vision-language-action model, lerobot checkpoint: Florence-2 encoder + soft-prompted flow-matching transformer) running on one Tenstorrent Blackhole p150a via tt-nn: 3 camera views + an instruction + proprio state in, a 30-step x 20-D action chunk out. Weights: lerobot/xvla-base · Paper: arXiv:2510.10274 · Upstream code: 2toinf/X-VLA · Port: changh95/tt-XVLA

Runs on p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  changh95/xvla-base-p150 --with-weights
tt-model serve changh95/xvla-base-p150

• Weights lerobot/xvla-base at cdb7964e4fe8 go to your HF cache; the image does not contain them.
• Serves on port 20000 (or the next free port); ready when the log says Application startup complete.

Run with tt-cli

tt serve changh95/xvla-base-p150
IMG=$(base64 -w0 media/pusht_synthetic.png)
printf '{"images":["%s","%s","%s"],"instruction":"push the T","state":[0,0,0,0,0,0,0,0],"seed":42}' "$IMG" "$IMG" "$IMG" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
tt model stop changh95/xvla-base-p150

• POST /predict: images (1 or 3 base64 PNG/JPEG views; 1 is copied into all 3 slots), instruction (or task); optional state (1–20 floats, default 8 zeros), domain_id (0), num_denoising_steps (1, max 50), seed.
• GET /health, GET /info, POST /reset.

Response

{"actions": [[-0.041, -0.11, 0.256, 0.146, -0.196, 0.145, 0.024, -0.016, -0.414, 0.406, ...], ...],
 "chunk_size": 30, "n_action_steps": 30, "action_dim": 20, "action_space": "ee6d", "normalized": false,
 "num_denoising_steps": 1, "seed": 42, "domain_id": 0, "instruction": "push the T", "language_tokens": 32,
 "state_dim": 8, "views_used": ["image", "image2", "image3"], "single_view_replicated": false,
 "input_size": [224, 224], "image_sizes": [{"width": 256, "height": 256}, ...],
 "timing_ms": {"preprocess": 2.47, "inference": 201.81, "total": 204.28}}

• actions is 30 rows of [x, y, z, r1..r6 (6-D rotation), gripper, 0 x 10] in raw model space (normalized: false): the base checkpoint ships no dataset statistics, so the values are not directly usable on a robot without fine-tuning.
• Views are ImageNet-normalised and resize_with_pad-ed to 224x224; the instruction is tokenised with the vendored facebook/bart-large tokenizer to a fixed 32 tokens.

Demo

Input (media/pusht_synthetic.png, synthetic push-T scene, sent as all 3 views)

Accuracy and speed

Metric	Value
Action-chunk PCC vs fp32 torch (5 seeds, 10 denoising steps)	0.999983 (max abs err 4.57e-03)
Open-loop action MAE vs fp32 torch (lerobot/pusht_image, 10 samples, 10 denoising steps)	253.78 vs 253.78 (delta +4.27e-04, +0.00%)
Inference, served over HTTP (warm, 3 views, 1 denoising step)	~194–211 ms inference · ~197–213 ms end-to-end per 30-step chunk
Inference, served over HTTP (warm, 3 views, 10 denoising steps)	~342 ms inference · ~345 ms end-to-end

Caveats

• Fixed shapes: 3 views at 224x224, 32 language tokens, batch 1; one request at a time (a lock serialises the device); TT_MESH_SHAPE must be 1x1.
• Default num_denoising_steps=1 is a speed setting (upstream uses 10); it costs <0.05% open-loop PCC but closed-loop flow-matching policies usually want 4–10 steps. Override per request.
• This is the BASE checkpoint, meant for fine-tuning: outputs are raw model-space actions, not normalised to any robot or dataset.
• Not an OpenAI-compatible API; GET /v1/models is a stub so the tt-model ready card does not 404.
• Validated on tt-metal v0.71.0-dev20260509-4 (main 2a6ddd8e572) with lerobot 0.5.0 / transformers 5.4.0, single p150a only.

Licensing

• Weights: lerobot/xvla-base, Apache-2.0; fetched from the Hub at serve time, not redistributed here.
• Port and serving code (code/): Apache-2.0, from changh95/tt-XVLA; it patches the lerobot 0.5.0 X-VLA policy (Apache-2.0), and code/tt/assets/bart-large-tokenizer/ vendors the facebook/bart-large tokenizer files (Apache-2.0).

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	2a6ddd8e572bb09b236a2adbd3afab1153e0a17e
code/ digest	f46cae534c35e55b (sha256, first 16 hex digits)
built	2026-09-12T13:19:29+00:00 by tt-model 0.1.0