flux2-dev-qb2

A tt-model container package: the serving platform ships as a Docker image, so a consumer needs only Docker and a Tenstorrent card — no tt-metal, no vLLM, no venv on the host.

Serve it

tt-model pull  stisiTT/flux2-dev-qb2
tt-model serve stisiTT/flux2-dev-qb2

Quickstart

FLUX.2 [dev] generating 1024x1024 images on four Blackhole chips, served over HTTP.

Before you start

The weights are gated. They are a pointer, not part of this image, so pull fetches them into your own HF cache under your own token. Accept the licence at black-forest-labs/FLUX.2-dev and run huggingface-cli login first, or the pull fails partway with a 403.

Budget roughly 210 GB of disk and a long first pull, split about evenly:

what	size	where
image	1.2 GB	docker
weights	~106 GB	your HF cache
weights converted to device layout	~105 GB	~/.cache/tt-model/flux2-dev-qb2/weights

The third one is written on first boot so later boots skip the conversion: 291s to ready the first time, 80s after.

You do not have to keep both copies. Once the converted cache exists, the pipeline reads the source weights only on a cache miss, so deleting everything except config.json, scheduler/ and tokenizer/ from the HF snapshot (~17 MB) leaves a working ~105 GB install that still boots in 80s. Keep the source if you intend to change resolution, mesh or precision: the cache is keyed by checkpoint, parallel config, mesh shape and dtype, and a change to any of them means fetching it again.

Hardware: four Blackhole chips, opened as a 2x2 mesh (p300x2 / QB2). This is not a preference. FLUX.2 needs its sequence and tensor parallel factors both above 1, so a four-chip line cannot run it, and the server refuses such a mesh at startup rather than producing images that are quietly wrong.

Generate an image

tt-model serve stisiTT/flux2-dev-qb2

# wait for readiness (first boot converts weights to device layout: several minutes)
curl -s localhost:8000/health

JOB=$(curl -s -X POST localhost:8000/generate \
        -H 'Content-Type: application/json' \
        -d '{"prompt": "a lighthouse in a thunderstorm", "seed": 3}' \
      | python -c 'import sys,json; print(json.load(sys.stdin)["job_id"])')

curl -s localhost:8000/jobs/$JOB                 # queued | running | done | error
curl -s -o out.png localhost:8000/jobs/$JOB/image

API

route	purpose
GET /health	readiness, model id, mesh shape, served resolution
POST /generate	enqueue a prompt; returns 202 and a job_id
GET /jobs	every job this process knows about
GET /jobs/{id}	one job's status
GET /jobs/{id}/image	the finished PNG
POST /jobs/{id}/cancel	cancel a job that has not started

POST /generate takes prompt (required), plus optional num_inference_steps, guidance_scale and seed.

Worth knowing

Generation is serialized. One device, one warm pipeline, one worker: concurrent requests queue rather than collide, and the API stays responsive while one runs.

The served resolution is fixed at startup. The pipeline builds its position embeddings and captures its trace for one shape, so a request asking for a different height/width is refused instead of silently rebuilding the pipeline. Serve a second instance with FLUX2_HEIGHT / FLUX2_WIDTH to offer another size.

Finished jobs and their PNGs are reaped an hour after completion. Download what you want to keep.

First boot converts weights to device layout and caches the result: measured 291s to ready on the first start and 80s on every start after that.

Serve profiles

One image serves every profile below; pick one with --profile.

profile	hardware	mesh	max_num_seqs	max_model_len
default (default)	p300x2	QB2

What is inside

• weights: black-forest-labs/FLUX.2-dev — downloaded to your HF cache at pull time, never baked into the image
• arch: blackhole
• serving stack: tt-dit-server

Provenance

Everything below is pinned; the image was built from exactly these.

component	pinned to
tt-metal	b28416bb4f010b558a31de8f46d7710cd53c7a4c (dirty tree)
code digest	3c35026f808ab477
built	2026-08-28T23:44:03+00:00 by tt-model 0.1.0

Shipped code

code/ in this repo is byte-identical to what runs inside the image.

• models/common/utility_functions.py
• models/tt_dit/blocks/
• models/tt_dit/encoders/
• models/tt_dit/layers/
• models/tt_dit/models/
• models/tt_dit/parallel/
• models/tt_dit/pipelines/
• models/tt_dit/utils/
• models/tt_dit/server/flux2/