olmo-3.1-32b-instruct
AI2's Olmo 3.1 32B Instruct (Apache-2.0), a fully open 32B dense decoder with sliding-window/full hybrid attention and 65k context, served on Tenstorrent Blackhole via vLLM with an OpenAI-compatible API including native tool calling. Profiles: p150 (one chip, bfp4 MLP, 16k context), p300 (TP2, 64k), p300x2 (TP4, 64k).
Runs on p150 or p300 or p300x2 — see the serve profiles below.
Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).
Quickstart
tt-model pull jashansinghTT/olmo-3.1-32b-instruct-blackhole --with-weights
tt-model serve jashansinghTT/olmo-3.1-32b-instruct-blackhole
pull --with-weights downloads the Docker image and the allenai/Olmo-3.1-32B-Instruct weights at ac0587e4a7744a551c059d8cd17ba220bc940dae (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.
Use with your client
curl -s localhost:20000/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"model":"allenai/Olmo-3.1-32B-Instruct","messages":[{"role":"user","content":"Hello!"}],"max_tokens":64}'
Tool calling works out of the box (tools in the request; finish_reason: tool_calls).
Serve profiles
One image serves every profile below; pick one with --profile.
| profile | hardware | mesh | max_num_seqs | max_model_len |
|---|---|---|---|---|
p150 (default) |
p150 | P150 | 32 | 16384 |
p300 |
p300 | P300 | 32 | 65536 |
p300x2 |
p300x2 | P300x2 | 32 | 65536 |
Provenance
The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:
| component | built from |
|---|---|
| tt-metal | a local checkout — commit not published |
| vLLM | v0.26.0 |
| vllm-tt-plugin | a local checkout — commit not published |
code/ digest |
e6c16e63163cbae6 (sha256, first 16 hex digits) |
| built | 2026-09-09T09:43:53+00:00 by tt-model 0.1.0 |