justintime47/FLUX.2-klein-9B-iQ4.5

FLUX.2 [klein] 9B — iQ 4.5

MLX only. An imatrix-calibrated conversion of black-forest-labs/FLUX.2-klein-9B for mlx-serve on Apple Silicon. The weights are MLX's packed affine format with a trimmed text encoder, so diffusers, ComfyUI and GGUF runtimes cannot load them.

Requires the next mlx-serve release, the first with calibrated image packs. Earlier builds refuse this pack at load: it keeps some transformer layers at full precision, which their FLUX loader does not accept.

	this pack	BF16 release
total on disk	8.56 GB	34.7 GB
transformer	5.11 GB at 4.5 bits per weight	18.2 GB
text encoder	3.28 GB at 4.5 bits per weight, 27 of 36 layers	16.4 GB
VAE	0.17 GB, bf16	0.17 GB

Bits per weight count the stored scales and biases, so 4.5 is the size of plain 4-bit group-64 quantization.

How it was made

• Calibration from the engine itself. Per-input-channel activation statistics were collected inside mlx-serve while it generated 73 varied prompts (portraits, landscapes, illustration styles, product shots, lettering including Korean and Japanese signage) across four aspect ratios at 4 steps.
• Weighted scale search. Each quantization group's scale and bias come from an activation-weighted least-squares search, with min/max only as the starting point.
• Measured bit allocation. Every linear's error was measured at 3, 4, 5, 6 and 8 bits, and the widths (3bit x39, 4bit x60, 5bit x45, bf16 x9) were bought by error saved per byte inside the budget.
• Conditioning tensors kept wide. The embedders, timestep embedder, the three modulation projections, the final norm and the output projection are held at bf16. Side-by-side renders at the same size showed cleaner detail and lettering than letting the allocator quantize them.
• Only the text encoder that runs. FLUX.2 [klein] reads Qwen3 hidden states 9, 18 and 27, so layers 27-35 and the LM head never run and are not shipped.
• Text-encoder MLP outputs kept wider. down_proj in layers 0-3, 6, 9, 12, 15, 18, 21 and 24 is held at 6-bit or wider, the layers llama.cpp's Q4_K_M widens. At the same size it rendered closer to the high-precision reference on 6 of 8 held-out prompts than the same pack without it.
• VAE untouched at bf16. No AWQ-style scale folding, so runtime LoRAs still compose with the stored weights.

Use

mlx-serve --serve --model /path/to/FLUX.2-klein-9B-iQ4.5 --port 11434
curl -s http://127.0.0.1:11434/v1/images/generations \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "a lighthouse on a rocky coast at dusk", "size": "1024x1024", "steps": 4}'

The MLX Core app lists it under the image generator's "On This Mac" models once it sits in a model folder.

License

Released by Black Forest Labs under the FLUX Non-Commercial License, included here as LICENSE.md. Non-commercial use only, subject to BFL's Acceptable Use Policy.