gpt-oss-20b-p150
A tt-model container package: the serving platform ships as a Docker image, so a consumer needs only Docker and a Tenstorrent card — no tt-metal, no vLLM, no venv on the host.
Capabilities: tool calling — yes (openai); reasoning parsing — yes (openai_gptoss).
Serve it
tt-model pull tt-hous/gpt-oss-20b-p150
tt-model serve tt-hous/gpt-oss-20b-p150
Quickstart
This package serves openai/gpt-oss-20b
as an OpenAI-compatible vLLM endpoint on either one P150 chip or two P150
chips on the same P300C board. It supports GPT-OSS reasoning, OpenAI-format
tool calls, and the full 131,072-token context window.
Select a serve profile
tt-model serve tt-hous/gpt-oss-20b-p150 --profile p150 --follow
tt-model serve tt-hous/gpt-oss-20b-p150 --profile p150x2 --follow
The first start compiles kernels and converts model tensors into the persistent package cache. Subsequent starts reuse both caches. Measured readiness was approximately 228 seconds cold / 65.5 seconds warm on P150 and 207.6 seconds cold / 52.3 seconds warm on P150x2.
Endpoint performance
Measured with vllm bench serve against the packaged image. Requests used
exact random input/output lengths, concurrency 1, greedy sampling, EOS
ignored, and one unmeasured warmup request per shape. Values are p50.
decode t/s/u is decode tokens per second per active user, derived as
1000 / p50 TPOT; E2EL is measured full-request latency.
P150
| ISL | OSL | concurrency | decode t/s/u | TTFT (ms) | E2EL (ms) |
|---|---|---|---|---|---|
| 128 | 128 | 1 | 17.34 | 545.5 | 7,870.8 |
| 512 | 128 | 1 | 17.26 | 4,284.0 | 11,641.9 |
| 1,024 | 128 | 1 | 17.19 | 4,285.2 | 11,673.3 |
| 2,048 | 128 | 1 | 17.16 | 8,572.4 | 15,973.8 |
| 4,096 | 128 | 1 | 17.01 | 17,131.8 | 24,599.0 |
| 8,192 | 128 | 1 | 16.93 | 34,289.6 | 41,794.7 |
| 8,192 | 1,024 | 1 | 16.95 | 34,294.1 | 94,660.3 |
| 16,384 | 128 | 1 | 16.72 | 68,808.9 | 76,406.4 |
| 32,768 | 128 | 1 | 16.45 | 138,597.6 | 146,317.1 |
P150x2
| ISL | OSL | concurrency | decode t/s/u | TTFT (ms) | E2EL (ms) |
|---|---|---|---|---|---|
| 128 | 128 | 1 | 29.91 | 307.5 | 4,552.6 |
| 512 | 128 | 1 | 29.68 | 2,267.7 | 6,546.8 |
| 1,024 | 128 | 1 | 29.74 | 2,270.1 | 6,542.0 |
| 2,048 | 128 | 1 | 29.58 | 4,562.4 | 8,855.7 |
| 4,096 | 128 | 1 | 29.40 | 9,110.6 | 13,430.4 |
| 8,192 | 128 | 1 | 29.23 | 18,237.3 | 22,582.6 |
| 8,192 | 1,024 | 1 | 29.26 | 18,239.3 | 53,205.0 |
| 16,384 | 128 | 1 | 29.01 | 36,576.9 | 40,954.7 |
| 32,768 | 128 | 1 | 28.97 | 73,654.7 | 78,038.3 |
The GPT-OSS expert decode path is batch-one, so both profiles intentionally
set max_num_seqs=1. P150x2 sustains about 29 decode tokens/s/user versus
17 on P150 and cuts long-prefill latency by approximately 47%.
Validation
Both profiles passed /health, /v1/models, deterministic chat, separated
reasoning/final content, OpenAI-format tool calling, clean shutdown, and an
exact 131,071-input-token plus one-output-token context probe. A 30-token
teacher-forced comparison against the Hugging Face reference passed at
100% top-1 and 100% top-5.
Serve profiles
One image serves every profile below; pick one with --profile.
| profile | hardware | mesh | max_num_seqs | max_model_len |
|---|---|---|---|---|
p150 |
p150 | P150 | 1 | 131072 |
p150x2 (default) |
p150x2 | P150x2 | 1 | 131072 |
What is inside
- weights:
openai/gpt-oss-20bat6cee5e81ee83917806bbde320786a8fb61efebee— downloaded to your HF cache at pull time, never baked into the image - arch: blackhole
- serving stack:
vllm-plugin
Provenance
Everything below is pinned; the image was built from exactly these.
| component | pinned to |
|---|---|
| tt-metal | 6ea930fd3d5dee17a076916eda2cdc2d8e5ba380 (dirty tree) |
| vllm | None |
| plugin | 053c0782aa11028924c21cb061ffa76576705cad |
| code digest | 279afa0f8bb2e8d7 |
| built | 2026-09-04T01:24:20+00:00 by tt-model 0.1.0 |
Shipped code
code/ in this repo is byte-identical to what runs inside the image.
models/common/models/demos/deepseek_v3/models/demos/deepseek_v3_d_p/models/demos/gpt_oss/models/tt_transformers/