devstral-small-2-24b-instruct-2512

Devstral Small 2 is Mistral's 24B agentic coding model, tuned for software engineering tasks: exploring a repository, editing multiple files, and driving tools. This package serves the text decoder on Tenstorrent Blackhole through vLLM, with tool calling enabled. The Pixtral vision tower is not ported, and a single die is not supported -- two dies is the minimum.

Runs on p300 or p150x4 or p300x2 — see the serve profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  anirud/devstral-small-2-24b-instruct-2512 --with-weights
tt-model serve anirud/devstral-small-2-24b-instruct-2512

pull --with-weights downloads the Docker image and the mistralai/Devstral-Small-2-24B-Instruct-2512 weights at 55c5b41e98c2dbd21b0c8afffc540dcfc9eb5128 (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Point a client at it

Once the server reports ready, the OpenAI-compatible endpoint is on the port tt-model serve printed (20000 by default).

tt-model serve anirud/devstral-small-2-24b-instruct-2512 --profile p150x4
tt-model curl "Refactor this function to use a generator."

Tool calling is on: send an OpenAI tools payload and the reply carries finish_reason: tool_calls.

Profiles

profile	dies	notes
p300	2	one p300 card
p150x4	4	the default
p300x2	4	the same four dies under the QB2 label

There is no single-die profile. The weights and KV cache fit in one 32 GB die, but a prefill matmul overruns L1: tt_transformers picks the same MLP prefill grids on one die as on four, so one die's cores carry the whole MLP rather than a quarter of it.

Not ported

The Pixtral vision tower. This is a text-only port: image inputs are not supported.

Serve profiles

One image serves every profile below; pick one with --profile.

profile	hardware	mesh	max_num_seqs	max_model_len
p300	p300	P300	4	32768
p150x4 (default)	p150x4	P150x4	8	65536
p300x2	p300x2	P300x2	32	65536

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	a local checkout — commit not published (dirty tree — the image includes uncommitted changes)
vLLM	v0.26.0
vllm-tt-plugin	a local checkout — commit not published
code/ digest	71a2d227e4520616 (sha256, first 16 hex digits)
built	2026-09-12T02:12:53+00:00 by tt-model 0.1.0