MOSS-Image (TTNN · Blackhole)

OpenMOSS MOSS-Image, a flow-matching text-to-image diffusion model, ported to run on a single Tenstorrent Blackhole p150 card in pure ttnn (no tt-metal source tree, no vLLM). Packaged as a v6 "thin" tt-dit-server bundle: tt-model pull builds a self-contained per-model venv from pip pins plus a small bundled wheel, and tt-model serve runs an HTTP image API. Experimental community bring-up.

At a glance

• Architecture: flow-matching DiT (~5B, single-stream in-context, 42 layers) + a Qwen3.5-9B Gated-DeltaNet hybrid text encoder + the FLUX.2 small VAE decoder.
• Status: Experimental community bring-up (not a Tenstorrent-verified model).
• Hardware: 1× Blackhole p150 + firmware/driver. SFPI is auto-provisioned on first serve (needs network once). No tt-metal install on the host.
• Engine: ttnn==0.80.0 (the TT-Metalium runtime rides inside the ttnn wheel).

Intended use

Text-to-image generation (and layout-controlled generation via MOSS-Image's structured-JSON prompts) for research and evaluation on Blackhole hardware.

Out of scope: the model is bf16 (no quantized build). Single-chip p150 is the default, validated path; a 1×N multi-chip tensor-parallel DiT is optionally available (see below).

Quickstart

On any host with a Blackhole p150, firmware, and network access:

pip install git+https://github.com/tenstorrent/tt-model-manager   # provides `tt-model`
tt-model pull  mando2222/moss-image
tt-model serve mando2222/moss-image            # OpenAI-style image API on :8000

First serve provisions SFPI (~one-time download) and JIT-warms kernels before it logs Application startup complete; subsequent serves are fast. The weights (OpenMOSS-Team/MOSS-Image-Base, ~26 GB bf16) are a pointer and are fetched into the bundle's hermetic HF cache on first serve (tt-model pull --with-weights to pre-fetch).

Using it (HTTP API)

The server is a plain ASGI app (uvicorn), bound to 0.0.0.0:$PORT (default 8000), reachable from other hosts. Diffusion → no tokens/KV-cache; the API is image-oriented.

# async job (recommended)
JID=$(curl -s -X POST http://<host>:8000/generate -H 'content-type: application/json' \
  -d '{"prompt":"a red fox in snow","height":1024,"width":1024,"num_inference_steps":50,"seed":42}' \
  | python3 -c 'import sys,json;print(json.load(sys.stdin)["job_id"])')
curl -s http://<host>:8000/jobs/$JID                      # poll until status":"done"
curl -s http://<host>:8000/images/$JID.png -o out.png     # fetch the image

# OpenAI-compatible (blocking)
curl -s -X POST http://<host>:8000/v1/images/generations -H 'content-type: application/json' \
  -d '{"prompt":"a red fox in snow","size":"1024x1024"}'

Full endpoint reference: see API_SPEC.md in the companion repo / the Discussions tab.

Performance

metric	value
End-to-end image PCC vs CPU reference (256², 4 steps)	0.99735
Text-encoder last_hidden_state PCC vs HF reference	0.9997
DiT full 42-layer forward PCC vs reference	0.99986
Latency, 256² / 4 steps	~25 s
Latency, 1024² / 50 steps	~235 s
Bundle warm serve, 256² / 4 steps	~12.5 s

Measured on a single Blackhole p150 (node4), ttnn==0.80.0, bf16 weights. The device VAE (now default) matches the host fp32 decoder to PCC ≥0.9999 at all resolutions and is faster at scale (~1.4 s vs ~6.6 s host @1024² decode).

Limitations

• VAE now decodes on the device (pure ttnn) by default — validated against the host diffusers fp32 decoder at all resolutions (device-vs-host image PCC 0.99994 @256² & 512², 1.0000 @1024² and @1344×768 non-square; MAD 1.3e-3) and faster at scale (1.4 s vs ~6.6 s host @1024²). Force the host fp32 decoder with MOSS_VAE_DEVICE=0. (The earlier high-res corruption — a per-input-shape conv-weight-preprocessing bug and a fixed group-norm core grid — is fixed.)
• SFPI is fetched over the network on first serve; an air-gapped host must pre-stage <ttnn>/runtime/sfpi.
• bf16 only (no quantized build). Single-chip is the default; multi-chip is optional (see below).
• tt-model package-thin (v6 thin) is upstream BETA.

Running on 1 / 2 / 4 cards

The DiT can run tensor-parallel across a 1×N Blackhole mesh; pick the topology with the MOSS_MESH env. The text encoder and VAE stay single-die. open_mesh enables FABRIC_1D automatically — the caller just needs an N-chip lease with device visibility fenced to exactly those dies. /health reports the active mesh_shape.

Reset the target dies before a mesh serve: tt-smi -r <dies> (e.g. tt-smi -r 0 1 for 2-card, tt-smi -r 0 1 2 3 for 4-card). Fabric firmware needs clean silicon — after single-chip workloads the ethernet handshake times out (Fabric Router Sync: Timeout) until the chips are reset. Reset again if a mesh server ever restarts. (run.sh does not auto-reset.)

1 card — default, turnkey:

tt-model serve mando2222/moss-image            # MOSS_MESH=1x1 (single chip)

2 cards — 1×2 mesh (lease 2 chips, then serve with both visible):

MOSS_MESH=1x2 TT_METAL_VISIBLE_DEVICES=0,1 MOSS_L1_SMALL=40960 \
  bash <install>/run.sh

run.sh honors a pre-set TT_METAL_VISIBLE_DEVICES; without the override it fences to one die. The two visible dies must be the intended linked pair — on a 4-die box, fencing to the wrong two makes fabric auto-discovery build a 4-chip mesh and the 2-chip sub-mesh fails the eth handshake.

4 cards — 1×4 mesh (on a 4-die box, e.g. a 2× p300c quietbox):

MOSS_MESH=1x4 TT_METAL_VISIBLE_DEVICES=0,1,2,3 \
  bash <install>/run.sh

Which counts are available depends on the box:

• 2× p150 dev box (two link-cabled cards): 1 or 2 cards.
• 4-die box (2× p300c): 1, 2, or 4 — validated end-to-end there: 2.86× @ 4 chips (1024²/50, DiT-only; ~1.8× @ 2), output PCC ≥0.987 vs single-chip at 1/2/4.

Honesty note: single-chip is the fully-validated turnkey serve path. The multi-chip DiT is validated at the pipeline level on a 4-die box; serving it through the bundle uses the env above and needs an N-chip lease. Small images (256²) are overhead-bound and don't scale — the win is on compute-bound full-size generations.

HTTP API contract

• Async jobs with live progress: POST /generate → job_id; GET /jobs/{id} reports stage (encoding → loading_transformer → denoising → decoding → complete), current_step, progress_percent, and eta_s.
• Public jobs persist their PNG to disk and survive a restart; list with GET /jobs, fetch with GET /images/{id}.png, and remove with DELETE /jobs/{id} (deletes the file too).
• Private ephemeral jobs: pass an X-Moss-Job-Token header on POST /generate and the job is token-scoped — never listed in GET /jobs, never persisted to disk, and its result is held in memory only, reaped after a 900 s TTL (private_result_ttl_s in /health). Fetching a private job's status/image requires the same token.
• /health advertises the contract: vae_device, mesh_shape, progress_reporting, job_persistence, private_ephemeral_jobs, private_result_ttl_s.
• OpenAI-compatible blocking endpoint: POST /v1/images/generations (accepts the full control set, not just prompt/size/seed).
• Generation controls on both endpoints: num_inference_steps, guidance_scale, negative_prompt, seed, width, height, plus MOSS-Image knobs shift (scheduler shift), max_sequence_length, and skip-layer guidance (uncond_skip_layers, slg_layers, slg_scale, slg_start, slg_stop). GET /v1/parameters (alias /config) returns the schema with ranges so a UI can render controls dynamically. Use ≥28 steps for quality (few-step outputs look soft).

Risks

Few-step / low-resolution outputs are rough; use ≥28 steps for quality. Numerics are bf16 on device vs fp32 reference (PCC ~0.997 end-to-end) — results are close to, not identical to, the reference pipeline.

Licensing

• Weights (OpenMOSS-Team/MOSS-Image-Base): Apache-2.0.
• Port + serving code (ttnn_moss, moss_image_app.py): Apache-2.0.
• moss_gdn (bundled Gated-DeltaNet Python wrapper): Apache-2.0; the compiled gated-delta-rule kernels ship inside ttnn.

How it's built (self-containment)

v6 thin bundle: ttnn==0.80.0 + diffusers/transformers (scheduler, tokenizer, VAE weights + host fallback) + the moss_gdn ops-wheel (installed by path), plus the ttnn_moss model code (text encoder + DiT + device VAE + optional tensor-parallel DiT). No tt-metal source tree, no vLLM, weights as a pointer. SFPI self-provisions at first serve.