tt-hous/ornith-1.5-9b

ornith-1-5-9b

Ornith-1.5-9B ported to Tenstorrent Blackhole, served through upstream vLLM with the standalone Tenstorrent plugin. One image, three profiles: p150 (one ASIC), p150x2 (two), p150x4 (four), each at the full 262144-token context. This package does NOT meet its release gates — GPQA scores 16/23 against a required 17 under the disclosed acceptance protocol (greedy decoding with the model card's presence_penalty 1.5; see Known limitations), and none of the six original time-to-first-token rows are within their latency limits. IFEval (81.97%) and the OpenAI API conformance suite pass on p150x4. It is published in that state for evaluation, not production.

Runs on p150 or p150x2 or p150x4 — see the serve profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  tt-hous/ornith-1.5-9b --with-weights
tt-model serve tt-hous/ornith-1.5-9b

pull --with-weights downloads the Docker image and the ornith-ai/Ornith-1.5-9B weights at 489cb97981b8654bcfcf30ce1f94ed1b62e07b53 (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Serve it

tt-model serve tt-hous/ornith-1.5-9b                  # default: p150x4
tt-model serve tt-hous/ornith-1.5-9b --profile p150   # one ASIC
tt-model serve tt-hous/ornith-1.5-9b --profile p150x2 # two ASICs

tt-model profiles tt-hous/ornith-1.5-9b lists them. Point any OpenAI-compatible client at the port tt-model serve prints.

What was measured, and on which profile

Gate	Result	Profile
OpenAI API suite	PASS 12/12	p150, p150x2, p150x4
meta_ifeval	PASS 81.97% (four-metric mean; proxy 23 ≥ 22)	p150x4
meta_gpqa_cot	FAIL 16/23 (69.57%), gate 17/23	p150x4
meta_ifeval	PASS 80.47% (four-metric mean; proxy 23 ≥ 22)	p150
meta_gpqa_cot	FAIL 13/23 (56.52%), gate 17/23	p150
TTFT benchmarks	FAIL 0/6 rows	p150x4 only

Measured 2026-09-17 under the acceptance protocol described below (presence_penalty=1.5), with the block-table determinism fix in place, so every figure is reproducible. p150x4 serves attention HiFi2 + bfloat16 KV cache; p150 serves the previous gate/up HiFi2 policy (bfp8 KV), the only one that fits one ASIC, and its single-chip decode path is numerically its own — hence the separate rows. The superseded frozen-greedy, zero-penalty figures on p150x4 were IFEval 86.48% and GPQA 11/23.

p150x2 is validated as working — it serves and passes the API suite — but carries no accuracy measurements. Do not assume another profile's numbers transfer to it.

Measured endpoint performance

Measured with vllm bench serve against the packaged server settings on this two-P300C host: one ASIC for p150, both ASICs of one board for p150x2, and all four ASICs across both boards for p150x4. Each row is the p50 of three sequential requests with exact random input/output lengths, concurrency 1, greedy sampling, and EOS ignored, taken after three discarded warm-up requests of the same shape (the first three requests of a new shape capture device traces and run several times slower), on the chat completions endpoint (whose template adds about ten tokens to each input). Decode tokens/s is derived as 1000 / p50 TPOT. p150 serves the previous precision policy (see Known limitations); p150x2 and p150x4 serve the promoted one.

P150

Input tokens	Output tokens	TTFT (ms)	Decode tokens/s	TPOT (ms)
128	128	110	6.04	165.56
2,048	128	349	6.00	166.74
8,192	128	1,183	5.87	170.31
32,768	128	5,697	5.42	184.55

P150x2

Input tokens	Output tokens	TTFT (ms)	Decode tokens/s	TPOT (ms)
128	128	84	9.75	102.54
2,048	128	279	9.69	103.19
8,192	128	974	9.51	105.10
32,768	128	4,884	8.87	112.78

P150x4

Input tokens	Output tokens	TTFT (ms)	Decode tokens/s	TPOT (ms)
128	128	77	15.48	64.58
2,048	128	172	15.40	64.95
8,192	128	590	15.18	65.89
32,768	128	2,729	14.35	69.70
65,536	128	6,851	13.38	74.75
131,072	128	19,380	11.78	84.88
262,016	128	61,184	9.53	104.88

The final P150x4 row uses 262,016 input tokens plus 128 output tokens, exactly filling the configured 262,144-token context window; it was measured on the completions endpoint, which adds no template tokens. Numbers are from the image this card ships with, taken 2026-09-17 (state/package-rebuild-v1/latency-warm/).

Known limitations

• Temperature-0 output is reproducible. A vllm-tt-plugin bug fixed 2026-09-16 (InputBatch.block_tables_for_rows passed block-table entries past each request's allocation) made identical requests non-reproducible and could corrupt the prompt. This image contains the fix. The fix did not change the GPQA or TTFT results.
• GPQA 16/23 under the acceptance protocol adopted 2026-09-17 — greedy decoding with the model card's presence_penalty=1.5, on the shipped attention: HiFi2 + bfloat16-KV precision. One short of the 17/23 gate; five of 23 rows still loop to the 32768-token budget. Under the earlier frozen greedy, zero-penalty protocol the same model scored 11/23. The protocol change costs one IFEval proxy point (86.48% → 81.97%, proxy 24 → 23, still passing) and about 8% TTFT. The card's own GPQA figure was measured under sampled decoding averaged over five runs; that protocol was piloted here and rejected (predicted ~13.6/23: it frees fewer hard rows than it costs elsewhere). 16/23 is the measured ceiling. Full record: state/gpqa-passing-v1/RESULTS.md.
• All six original TTFT rows exceed their limits on p150x4; concurrent-prefill latency is ~8-11% worse than the older fork-based stack.
• p150x2 was unusable before 2026-09-17. Its first request died in the two-ASIC decode warm-up with an L1 circular-buffer clash (in the 2026-09-15 published image as well): the decoder pinned the 24 recurrent states in L1 under a budget validated only on four chips. This image keeps them in DRAM on two chips (tt/multichip_decoder.py, STACK_STATE_L1_BUDGET); the profile serves and passes the API suite. Record: state/package-rebuild-v1/RECORD.md.
• p150 serves the previous precision policy (gate_up_hifi2_policy.json, bfp8 KV cache); p150x2 and p150x4 serve the promoted attn_hifi2_kv16_policy.json. The promoted policy's bfloat16 KV cache (~8.5 GB at the full 262144-token pool) does not fit one ASIC beside the weights: the first trace capture fails with a DRAM out-of-memory. p150 was validated under the previous policy (API, reasoning, tool calls, determinism) but has no accuracy measurements under any policy.
• Device logprobs exist only on the four-ASIC mesh. On p150 and p150x2 a logprobs request is rejected at admission with HTTP 400.
• Host sampling is refused on every profile: min_p, bad_words, logit_bias, allowed_token_ids, min_tokens, structured outputs and top_logprobs > 0 all return HTTP 400.

Serve profiles

One image serves every profile below; pick one with --profile.

profile	hardware	mesh	max_num_seqs	max_model_len
p150	p150	P150	32	262144
p150x2	p150x2	P150x2	32	262144
p150x4 (default)	p150x4	P150x4	32	262144

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	a local checkout — commit not published (dirty tree — the image includes uncommitted changes)
vLLM	v0.26.0
vllm-tt-plugin	a local checkout — commit not published (dirty tree — the image includes uncommitted changes)
code/ digest	552ceb242e43ca90 (sha256, first 16 hex digits)
built	2026-09-17T17:19:59+00:00 by tt-model 0.1.0