mimo-v2.6-distill-qwen-9b-tt-p150 on Tenstorrent p150

Derived from XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B by the Xiaomi MiMo team, unmodified. See the original model card for training and evaluation details.

MiMo-V2.6-Distill-Qwen-9B, a 9B-parameter hybrid Gated-DeltaNet / full-attention reasoning model (24 linear-attention and 8 full-attention layers, 262,144-token context) that Xiaomi MiMo fine-tuned from Qwen3.5-9B on MiMo-V2.6-generated data, served with vLLM. Text only; thinking is on by default and comes back separately from the answer. Tested at one user and at 32 concurrent users.

Runs on Tenstorrent p150 — see the serving profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Hardware	p150
Context	262,144 tokens
License	mit

Intended use

Reasoning chat, coding and agent-style text tasks through the OpenAI-compatible API: one interactive user at full speed (--profile single) or up to 32 concurrent users (the default profile).

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150

Quickstart

tt serve ndaly/MiMo-V2.6-Distill-Qwen-9B

tt serve downloads the Docker image and the XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B at 2367e865d009c13ac81713a2878291d33ab28177 weights (into your HF cache; they are not in the image), then starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve ndaly/MiMo-V2.6-Distill-Qwen-9B.

A first request, with the sampling settings the checkpoint recommends:

curl -s localhost:20000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B",
  "messages": [{"role": "user", "content": "Solve step by step: what is 17 * 23?"}],
  "max_tokens": 1024, "temperature": 0.6, "top_p": 0.95, "top_k": 20}'

Serving profiles

Select a different serving profile using --profile.

profile	hardware	mesh	max_num_seqs	max_model_len	description
batch32 (default)	p150	P150	32	262144	Up to 32 concurrent users (decode bucketing picks the trace for the active rows); the stage-11 32-user profile.
single	p150	P150	1	262144	One interactive user at full speed (max_num_seqs 1); the stage-11 single-user profile.

Capabilities

The server speaks the OpenAI API at http://127.0.0.1:20000/v1 (or whichever port your serve command reported; chat completions, completions, and /v1/models). Pass "model": "XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B" — the weights id, not this package's name.

Reasoning output is separated (--reasoning_parser mimo): the thinking text arrives in reasoning_content, apart from content.

The checkpoint's recommended sampling is temperature 0.6, top-p 0.95, top-k 20. The server closes a thinking block after 1,024 reasoning tokens (MIMO_THINK_BUDGET) so that every reply carries an answer, and a reply the model writes without think tags is returned as the answer.

Expected performance

eval	this package	reference	prompts
GSM8K-CoT, strict match (thinking on, 4,096-token budget)	79.3 %	—	256 of 1,319
GSM8K-CoT, flexible extract	85.2 %	—	256 of 1,319
IFEval, prompt-strict	65.2 %	—	256 of 541
IFEval, instruction-strict	74.3 %	—	256 of 541

Evals ran through the served endpoint at concurrency 32 with the recommended sampling. The original model card reports no GSM8K or IFEval figure.

profile	ISL	OSL	concurrency	N	TTFT (ms)	prefill tok/s/u	decode tok/s/u	E2EL (s)
single	4,096	128	1	—	461	8,890	44.9	3.3
batch32	4,096	128	32	—	14,900	275	13.1	24.7

Measured on p150 through the served endpoint on 2026-10-09 with the full 262,144-token context enabled. TTFT and time per output token (22.3 ms and 76.6 ms) are measured; prefill tok/s/u is ISL / TTFT, decode tok/s/u is 1 / TPOT and E2EL is TTFT + OSL x TPOT. At 32 users the 32 prefills run one after another, so TTFT covers the whole wave; aggregate output was 166 tok/s. The number of runs behind each row was not recorded.

Limitations

• Validated on p150 only (one Blackhole chip); no other board.
• Text only: image input, which the Qwen3.5 base supports, is not ported.
• No tool-call parser is configured, so tools in a request are not turned into tool_calls.
• Reasoning is capped at 1,024 tokens per reply; a longer chain is closed and the answer follows. A loop guard (MIMO_LOOP_GUARD) also stops a reply that starts repeating itself.
• Prefix caching is off, so every request prefills from scratch; at 32 users prefills run sequentially and TTFT for 4,096-token prompts is about 15 s.
• Weights run at reduced precision (bfp4 projections, bfp8 KV cache and LM head), not bf16; outputs differ from the Hugging Face reference.
• The upstream checkpoint is an SFT release that Xiaomi publishes as a research starting point; read the original card before relying on it in production.

Licensing

Weights: MIT (Xiaomi). Port and serving code in code/: Apache-2.0.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/ndaly/MiMo-V2.6-Distill-Qwen-9B/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	a local checkout — commit not published
vLLM	v0.29.0
vllm-tt-plugin	77fcb6e4a16be794cbe05748492cf1c51150db7e
code/ digest	fdfb1a1547e6d03d (sha256, first 16 hex digits)
built	2026-10-15T19:55:03+00:00 by tt-model 0.1.0