mlx-community/flux-3-image-klein-bf16

This FLUX Model is licensed by Black Forest Labs Inc. under the FLUX Kommunity License. Copyright Black Forest Labs Inc. IN NO EVENT SHALL BLACK FOREST LABS INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.

This repo is an Apple Silicon copy of black-forest-labs/flux-3-image-klein for the Swift/MLX port flux3-klein-swift.

• The files are unmodified. Every file here is byte-identical to upstream revision 33733762 (sha256 checked). MLX loads these bf16 safetensors directly, and the int8 / int4 tiers are made when the model loads.
• It is an independent port. It is not an official Black Forest Labs product, and BFL has not endorsed or validated it.

Licence: read it before you use this model

The DiT and autoencoder are under the FLUX Kommunity License v1.0:

• Non-commercial use is permitted.
• Commercial use of outputs is permitted only for a "Qualifying User": under US$5M gross annualized revenue, affiliates included. You must also filter content (or review outputs) and disclose AI generation where the law requires it (§2(a), §2(e)).
• No training on outputs: you may not use the model, its derivatives or its outputs to improve any other AI model that performs a similar function (§4(b)).

Read the full licence for the other restrictions and for termination. For any other commercial use, see bfl.ai/licensing. The text encoder (text_encoder/, byte-identical to Qwen/Qwen3-VL-4B-Instruct) is Apache-2.0 (text_encoder/LICENSE).

Model

FLUX 3 Image Klein does text-to-image, single-reference editing and multi-reference composition (up to 10 references) in one model. Bounding-box layout goes in the prompt, and the model renders text.

• DiT (5.55B): each mode (image, references, text) has its own 5-block stack; then come 28 shared single-stream blocks with per-token modulation. It is a flow-map distill: 16 steps, no CFG.
• Conditioner: 8 hidden-state taps (layers 4…32) of Qwen3-VL-4B-Instruct, text only, giving 20480-d features.
• Autoencoder: the FLUX.2 VAE, the same weights as FLUX.2 [klein]. vae/ is the diffusers form the port loads; ae.safetensors is BFL's native form.

File	What
model.safetensors, config.json, manifest.json	the distilled DiT, bf16, in BFL's native key names (359 tensors)
text_encoder/	Qwen3-VL-4B-Instruct, bf16
vae/, ae.safetensors	the autoencoder, fp32
LICENSE.md, NOTICE.md	the FLUX Kommunity License and BFL's attribution notice

Prompting. The model expects a full scene caption (medium, scene, lighting, each subject's colours, materials and position). Run a prompt upsampler first: BFL's Apache-2.0 flux-3-image-klein-pup, or any LLM given its system_prompts.json. Layout boxes go after the caption as a JSON list on a 0–1000 grid; see the upstream card.

Sampling

Setting	Value
Sampler	flow-map Euler; t′ = α·t / (1 + (α − 1)·t) over linspace(1, 0, 17)
Steps	16
α	6.93 (config.json). BFL's reference CLI defaults to 16.0, which does not reproduce BFL's example renders (35.8 vs 23.1 dB on the knitted example)
Guidance	none

Parity (Swift/MLX port vs BFL's reference code, fp32)

Component	Result
DiT, one forward, all stages (t2i, 1 ref, 3 refs)	cos ≥ 0.9999998, relMax ≤ 2.1e-5
DiT, 16-step 512² trajectory	relMax ≤ 1.7e-4 at every step
Conditioner (8 taps, real and padded rows)	cos ≥ 0.99998
VAE encode / decode	cos 0.999999 / 0.9999999
Reference prep (fit, pad, crop)	byte-identical
bf16 end to end vs BFL's own example PNGs (same noise)	35.9 / 39.8 / 52.7 / 30.9 dB; BFL's code on MPS scores 35.8 / 42.5 / 56.2

Quant tiers, text fidelity and memory (M5 Max 128 GB)

The DiT tiers are made when the model loads: affine, group 64, block by block from the bf16 file. Embeddings, timestep MLPs, modulations and the final layer stay bf16.

• int4 keeps every block's attention projections at 8 bits. Plain int4 drifted too far from bf16.
• The text encoder loads text-only (the vision tower is never read). It can also be int8.

Text fidelity (an OCR gate: 20 long-text EN/ZH prompts × 2 seeds per lane, Apple Vision OCR, Δ against bf16 with a 95 % bootstrap CI; tolerance pre-registered at ΔCER ≤ +1 pp):

DiT + encoder	CER	ΔCER vs bf16	exact strings	Verdict
bf16 + bf16	3.00 % (EN 0.19 %)	—	68.7 % (EN 95.8 %)	baseline
int8 + bf16	2.61 %	−0.39 pp [−0.85, −0.05]	72.4 %	within tolerance
int4 (8-bit attn) + bf16	3.10 %	+0.10 pp [−0.69, +0.91]	71.6 %	within tolerance
bf16 + int8	2.95 %	−0.05 pp [−0.32, +0.21]	69.4 %	within tolerance
int8 + int8	3.05 %	+0.05 pp [−0.33, +0.46]	68.7 %	within tolerance

English text is excellent; Chinese is weak in every tier (12 % CER). int4 keeps text, but at a given seed it is a visibly different render from bf16 (21.9–35.5 dB on BFL's examples, against 31–50 dB for int8).

Memory (MLX with a 2 GB buffer-pool cap, phys_footprint peak of the whole process):

DiT + encoder	Resident (MLX)	Peak, 1024² T2I or edit	Peak, encoder dropped after encoding
bf16 + bf16	19.5 GB	31.8 GB (4 refs: 32.2 GB)	24.1 GB
int8 + bf16	14.5 GB	26.9 GB	—
int4 (8-bit attn) + bf16	12.7 GB	25.1 GB	—
bf16 + int8	16.1 GB	28.4 GB	—
int8 + int8	11.1 GB	23.5 GB	18.8 GB

• About 11 GB of each 1024² peak is the fp32 VAE decode.
• A single reference at the 4 MP cap (2048² canvas and output) peaks at 51–58 GB. It needs a 64 GB Mac.

Speed (16 steps, the same in every tier):

Request	Time
T2I	~1.0–1.2 s per step
One reference	~2.3 s per step
Four references	~10 s per step
4 MP single reference at 2048²	~20 s per step

Use (Swift)

import Flux3Klein

let pipe = try await Flux3KleinPipeline.load(snapshot: snapshotURL, quant: .bf16)   // or .int8 / .int4
let result = try pipe.generate(.init(prompt: caption, references: [try Flux3RGBImage.read(url: photoURL)]))
try result.image.write(to: outURL)

Code: https://github.com/xocialize/flux3-klein-swift (Swift/MLX port, MIT; the weights stay under the FLUX Kommunity License).