hidream-o1-p100a

HiDream-O1-Image (the full, undistilled model), ported to one Tenstorrent Blackhole p100a. Weights: HiDream-ai/HiDream-O1-Image (0b0901d) · Upstream code: HiDream-ai/HiDream-O1-Image · Port: SunandBean/tt-hidream-o1

Runs on p100a (mesh 1×1). Configuration: Tensix (worker) dispatch, 1 command queue. The Qwen3-VL-8B language model (36 layers) runs on the card; the vision tower, the sampler and the patch maths run on the host.

What this port adds — the single-chip port itself, and the two upstream patches that let HiDream's own model code import and run without CUDA or flash-attn. What it builds on — the changh95/qwen-image-2.1-p150 single-chip port (Apache-2.0) and HiDream's own model code (MIT), vendored under code/tt_hidream_o1/upstream/.

What this model is

HiDream-O1 is unusual: there is no VAE and no latent space. It works directly on 32×32 pixel patches. The architecture is the whole of Qwen3-VL-8B (language 36 layers at 4096, GQA 32/8, head 128, SwiGLU 12288, q/k RMSNorm, 3-axis interleaved mRoPE [24,20,20] at theta 5e6, plus a 27-layer vision tower) with three generation parts bolted on:

Part
x_embedder	32×32 pixel patch (3072) → 1024 → 4096
t_embedder1	timestep → 4096, written into the <|tms|> token's embedding slot
final_layer2	4096 → 3072, back to a pixel patch

The sequence is [chat template text … <|boi|><|tms|>] + [target image patches] + [reference patches]. Text tokens attend causally among themselves; the generation tokens see everything.

It is the undistilled model: 50 FlowUniPC steps (shift 3) at CFG 5.0 means 100 forward passes per image. That is the price of its quality.

Quickstart (Python)

Prerequisite: a tt-metal / ttnn environment. ttnn is not on PyPI.

hf download sunandbean/hidream-o1-p100a --include "code/*" --local-dir hidream-o1-p100a && cd hidream-o1-p100a
pip install -e code/                  # the Python API only
pip install -e "code/[server]"        # also the HTTP server (fastapi, uvicorn, pydantic)

from tt_hidream_o1 import HiDreamTT, open_device, close_device

dev = open_device()
try:
    model = HiDreamTT(dev)            # downloads HiDream-ai/HiDream-O1-Image@0b0901d into your HF cache
    image, _req, timing = model.generate("Three red apples and two green pears on a wooden table, still life photograph",
                                         width=1024, height=1024, seed=1234, snap=False)
    image.save("output.png")
finally:
    close_device(dev)

snap=False matters. The official pipeline runs find_closest_resolution, which always rounds the request to its 4–5 megapixel list (2048², 2560×1440, …) — ask for 1024² and you get 2048². snap=False generates the size you asked for.

Task	Text to image. An editing path exists but is not usable at normal sizes — see Caveats.
Size	Generated on a 32 px grid; a height like 1080 is produced at 1088 and center-cropped.
Steps	50 FlowUniPC steps (shift 3) at guidance_scale 5.0 — 100 forwards.
Output	(PIL.Image, Timing). Timing.values holds vision_s, prefix_s, lm_s.
Weights	HiDream-ai/HiDream-O1-Image pinned to 0b0901d99f200389e138c61946af1185f5f49a13, downloaded on first use. HIDREAM_SNAPSHOT overrides.

Serving (HTTP)

uvicorn tt_hidream_o1.server:app --host 0.0.0.0 --port 20000
curl -s localhost:20000/predict -H 'Content-Type: application/json' \
  -d '{"prompt":"A cozy cafe storefront with a wooden sign that says \"OPEN DAILY\", morning light","width":1024,"height":1024,"seed":1234}'

• POST /predict: prompt, optional width (1024), height (1024), seed (42), num_steps (50). Returns image (base64 PNG), width, height, model, weights_revision, license, seed and timing_ms. The server always passes snap=False.
• GET /health, GET /info.
• Contract errors — a reference image, a step count other than 50, a size off the 32-px grid — return 422 without touching the card.

Demo

1024×1024, seed 1234, 50 steps at CFG 5, generated on the p100a.

…a wooden sign that says "OPEN DAILY", morning light	Three red apples and two green pears on a wooden table

Close-up portrait of an elderly woman with silver hair laughing, window light, 85mm	Snow-capped mountains reflected in a still alpine lake at sunrise

Performance

Batch 1, 50 steps, CFG 5, measured 2026-09-26 on an idle p100a: Tensix (worker) dispatch, 1 command queue.

Metric	832×640	1024×1024	2048×2048
End to end	15.5 s	19.4 s	—
Language model (100 forwards)		18.5 s
One warm forward	0.114 s	0.179 s	1.094 s
Sequence length	548	1050	4122

1920×1080 takes about 47 s. The forward cost grows with the sequence length, which is the patch count: 2048² is 4122 tokens against 1050 at 1024², and its warm forward is 6.1× slower.

Across 10 different 1024² prompts the total was 19.41–19.59 s — the cost is almost entirely the 100 forwards, so it barely varies with the prompt.

Model load	22.7 s
Device DRAM after load	14.0 GB of 29.9 GB

From the published container

The same image anyone can tt-model pull, reached over its own HTTP server on an idle card (2026-10-09). Three requests at 1024², 50 steps, seed 42: 20.28 s, 19.56 s, 19.89 s — the language model accounts for 18.1–18.5 s of each, and device DRAM after load is 14.0 GB of 29.9.

There is no RTX 5070 Ti column here, and the reason is not that the card won. HiDream-O1-Image is published as a transformers checkpoint rather than a diffusers pipeline, so there is no reference pipeline to time against; and at 17B in bf16 it does not fit in 15.5 GB in any case. The CPU accuracy comparison above is what this port is checked against.

Accuracy

Against the official pipeline run on CPU (device_check.py, first-step and end-to-end):

Case	First-step PCC	Final image PCC
1024×1024 text to image	0.99980 / 0.99986	0.9673
2048×2048 text to image	0.99827 / 0.99946	0.9924
832×640 text to image	0.99878 / 0.99978	0.9705
2048×2048 edit, aspect kept	0.99841 / 0.98595	0.9865

Token ids and mRoPE positions match the official implementation exactly in every case.

Caveats

• Single card only. Mesh 1×1 with Tensix (worker) dispatch; no multi-chip scaling.
• Slow by design. The undistilled model needs 100 forwards per image. At 1024² that is 19 s — about 6× Z-Image-Turbo and 6× FLUX.2 klein on the same card.
• Editing is not offered. All six 512–640 px edits at the full setting came out as noise. diag_edit.py resolved why: the official code only works at its ~4 MP sizes, and the official CPU pipeline is also noise at 512. At 2048² the edit is correct, but one costs about 6 minutes, so this port exposes text to image only. The Python API still takes refs.
• snap=True is the upstream default and will silently change your size to the nearest 4–5 MP resolution. The server always passes snap=False.
• Generation is on a 32 px grid; other heights and widths are produced larger and center-cropped.
• Prompt following is not perfect: in the review above, a Korean watercolour-alley prompt produced a Korean food plate.

Licensing

• Weights: HiDream-ai/HiDream-O1-Image, MIT, revision 0b0901d99f200389e138c61946af1185f5f49a13. Not redistributed here.
• Port code (code/): Apache-2.0 (SPDX headers on the modules), from SunandBean/tt-hidream-o1.
• code/tt_hidream_o1/upstream/: model code vendored from the official HiDream-ai/HiDream-O1-Image repo at 2c2d29ff729e48f33e41f49edfdbd81d5ac103b4 (MIT), with two changes: use_flash_attn reads an environment variable and defaults off (as the upstream README instructs for hosts without flash-attn), and an unconditional torch.cuda.memory_allocated() is guarded behind DEBUG_MEM so the module imports on a CPU host.
• Derived from the changh95/qwen-image-2.1-p150 single-chip port (Apache-2.0).
• code/THIRD-PARTY.md lists every directory under code/ that came from somewhere else, with its upstream commit and where its licence text sits in this repo.

Run it as a container

Everything below is generated by tt-model package from the manifest this image was built from.

tt-model pull  sunandbean/hidream-o1-p100a --with-weights
tt-model serve sunandbean/hidream-o1-p100a

pull --with-weights downloads the Docker image and the HiDream-ai/HiDream-O1-Image weights at 0b0901d99f200389e138c61946af1185f5f49a13 (into your HF cache; they are not in the image). serve starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Request shape

POST /predict takes {"prompt": str, "width": int, "height": int, "seed": int} and returns {"image": base64 PNG, "width", "height", "model", "weights_revision", "license", "seed", "num_steps", "timing_ms"}. The model is undistilled: the step count is fixed at 50 and any other value is a 422. Generating below roughly 4 MP degrades sharply -- see Caveats on the model card.

Provenance

The exact sources the image was built from — the model code under code/ is byte-identical to what is inside the image (pyproject.toml, PYTHON.md and examples/ are packaging files this repo adds, and are not in the image):

component	built from
tt-metal	58648d44443223041148125843ca7c7d4780463e
code/ digest	0780258eac622ba3 (sha256, first 16 hex digits)
built	2026-10-09T17:54:50+00:00 by tt-model 0.1.0