gemma-4-12b-it-p150

Google Gemma-4-12B-it served on a single Blackhole P150 via the Tenstorrent vLLM plugin, in thinking mode. Experimental community bring-up: single-user (TP=1) with host sampling, so async scheduling is disabled. Accuracy: 71.7% GPQA Diamond (thinking, exact_match) on 99/198 docs vs Google's 78.8% on the full 198.

Runs on p150 or p150 — see the serve profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  stisiTT/gemma-4-12b-it-p150 --with-weights
tt-model serve stisiTT/gemma-4-12b-it-p150

pull --with-weights downloads the Docker image and the google/gemma-4-12B-it weights at 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7 (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Serve

tt-model pull  stisiTT/gemma-4-12b-it-p150 --with-weights
tt-model serve stisiTT/gemma-4-12b-it-p150            # default profile: p150-eval
tt-model serve stisiTT/gemma-4-12b-it-p150 --profile p150-smoke   # 4k smoke

First serve compiles kernels for your device (several minutes) and is ready when it logs Application startup complete. The endpoint is OpenAI-compatible.

Shutting down

tt-model stop stisiTT/gemma-4-12b-it-p150

This releases the device cleanly. Avoid docker kill / docker rm -f: a hard kill leaves the card unable to load a model until the host is rebooted, and a card reset is not enough.

Note on this SKU

This bundle targets one Blackhole P150 die (1x1 mesh), fabric disabled. On a p300c (two P150 dies linked over ethernet) GEMMA4_FABRIC=off is required and is baked into the default env.

Serve profiles

One image serves every profile below; pick one with --profile.

profile	hardware	mesh	max_num_seqs	max_model_len
p150-eval (default)	p150	P150	1	65536
p150-smoke	p150	P150	1	4096

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	7d54a363f0cb5ccb0fc0ff8a63fb35398a4ba0b4 (dirty tree — the image includes uncommitted changes)
vLLM	v0.26.0
vllm-tt-plugin	bef89e429e202caa38d3e3c8a24da4e8cc02405a
code/ digest	3bed9dc451aa0fb8 (sha256, first 16 hex digits)
built	2026-09-12T15:59:42+00:00 by tt-model 0.1.0