AtomicChat/FLUX-3-Image-Klein-GGUF

How to Run FLUX 3 Image Klein Locally

Black Forest Labs' FLUX 3 Image Klein as GGUF for stable-diffusion.cpp, built from the published BF16 weights with our own importance matrix and measured on one stand against every other Klein GGUF we could find. The renders and measurements are public.

• Everything to run Klein in stable-diffusion.cpp is here: the image model, the Qwen3-VL-4B text encoder (text_encoders/) and the autoencoder (vae/). The image model files also load in ComfyUI through ComfyUI-GGUF; see In ComfyUI. The prompt upsampler Klein expects is in FLUX-3-Image-Klein-pup-GGUF.
• AD-Q5_K stays 14% closer to the BF16 picture than unsloth's UD-Q5_K_XL at a smaller size, and AD-Q4_K 21% closer than their Q4_K_M. Where their file is better, the table below says so.
• The text encoder moves the picture as much as the image model does. Our AD-Q4_K encoder is 21% closer than the Q4_K_M most people use, at the same size.
• Looking for uncensored? The open Klein pipeline has no content filter. Our text encoder and prompt upsampler without refusals are in FLUX-3-Image-Klein-TE-Uncensored-GGUF and FLUX-3-Image-Klein-pup-Uncensored-GGUF; what they change on the pictures is in Content filtering.

---

FLUX 3 Image Klein is Black Forest Labs' 5.6B text-to-image and editing model: one model for generation, single-reference edits and up to ten reference images, distilled to 16 steps at CFG 1. This repo has the image model as GGUF for stable-diffusion.cpp, with the text encoder and the autoencoder next to it.

Pick a file

Each file renders the same 48 captions × 2 seeds from the same starting noise as the BF16 weights, and every image is compared with the BF16 image of the same caption and seed.

• LPIPS is a perceptual distance: lower is closer to the original picture, 0 is identical.
• vs. new seed puts it on a scale. 100% is how far the BF16 weights move when only the seed changes, which gives an unrelated picture of the same caption (LPIPS 0.530).
• text (zh) is the LPIPS on the eight captions that put Chinese text into the image.

File	Size on disk	LPIPS [95% CI]	vs. new seed	SSIM	PSNR	text (zh)
BF16	11.10 GB	0 (identical)	0%	1.000	—	0
Q8_0	5.99 GB	0.014 [0.011, 0.017]	3%	0.982	34.4 dB	0.011
AD-Q5_K	4.44 GB	0.038 [0.032, 0.044]	7%	0.954	28.8 dB	0.033
AD-Q4_K	3.65 GB	0.081 [0.069, 0.095]	15%	0.903	24.4 dB	0.070

• Q8_0 is the closest to the original. Pick it if it fits.
• AD-Q5_K is the pick at about 4.5 GB: the same picture, small details move.
• AD-Q4_K keeps the composition and the lettering; textures and small objects shift.
• Below 3.6 GB we do not publish a file of our own: at 3.3 GB and 2.6 GB unsloth's UD-Q3_K_XL and UD-Q2_K_XL are as good as or better than our best layouts.
• The BF16 weights render deterministically here: a second BF16 render of the same caption and seed is LPIPS 0.0004 away, so every number above comes from quantization.

How these compare with other GGUFs

Every file below was downloaded and rendered here with the same captions, seeds, binary and GPU.

File	Size on disk	LPIPS [95% CI]	Ours at that size or smaller
leejet Q8_0	6.15 GB	0.016 [0.012, 0.021]	Q8_0, 5.99 GB: 0.014
unsloth Q8_0	6.09 GB	0.015 [0.012, 0.021]	Q8_0, 5.99 GB: 0.014
unsloth Q6_K	4.80 GB	0.035 [0.029, 0.043]	AD-Q5_K, 4.44 GB: 0.038. Our 4.78 GB layout measured the same as theirs (0.035), so we did not publish it
unsloth UD-Q5_K_XL	4.48 GB	0.044 [0.037, 0.052]	AD-Q5_K, 4.44 GB: 0.038, 14% closer
unsloth UD-Q4_K_XL	3.93 GB	0.055 [0.047, 0.062]	AD-Q4_K, 3.65 GB: 0.081. Theirs is the better file at this size
unsloth Q4_K_M	3.67 GB	0.103 [0.093, 0.114]	AD-Q4_K, 3.65 GB: 0.081, 21% closer
leejet Q4_K	3.51 GB	0.126 [0.112, 0.140]	none
unsloth UD-Q3_K_XL	3.28 GB	0.116 [0.101, 0.130]	none. Our best 3.26 GB layout measured the same (0.114), so we did not publish it
Abiray Q4_K_M	3.27 GB	0.129 [0.114, 0.144]	none
unsloth UD-Q2_K_XL	2.62 GB	0.203 [0.182, 0.224]	none. Theirs is better than our best 2.59 GB layout (0.212)
Abiray Q3_K_M	2.55 GB	0.279 [0.260, 0.297]	none
leejet Q2_K	2.28 GB	0.413 [0.389, 0.435]	none

• Unsloth's UD files are strong below 5 bits. They move bits from the reference-image blocks, which text-to-image never runs, to the last single blocks. We tried the same moves at 3.9 GB (rows AD-Q4_K-v/-w/-x in the metrics repo, 0.076 to 0.079 on one seed against their 0.054) and did not catch UD-Q4_K_XL.
• Abiray's Q4_K_M and Q3_K_M, and leejet's files, are stable-diffusion.cpp's defaults (sd-cli -M convert --type): Abiray's measure the same as the plain Q4_K and Q3_K we built. The _K_M suffix does not mean a mixed layout there.
• Our Q8_0 is 0.1 GB smaller than the other Q8_0 files because the stable-diffusion.cpp version we used also quantizes the six modulation layers, and it is still closer to the original.
• Not measured: unsloth's Q5_K_M (4.22 GB) and Q3_K_M (3.01 GB), and leejet's and Abiray's other sizes, which are the tool's defaults like the ones above.

What AD- changes

AD- means Atomic Dynamic: the type is chosen per tensor from a sensitivity scan of this model, and the file is quantized with an importance matrix from 32 upsampled captions that share nothing with the evaluation set. The scan ranked, per byte saved, the six modulation layers, the attention v and output projections and the last 14 single blocks as the most sensitive, and the MLP input projections as the least.

• AD-Q5_K (base Q5_K): the modulation layers stay BF16; attention v and output Q8_0 in the last 14 single blocks and Q6_K elsewhere; everything else in the last 14 single blocks Q6_K; the five text-only blocks Q6_K; the five reference-image blocks, which run only for edits, Q3_K.
• AD-Q4_K (base Q4_K): the five text-only blocks Q6_K; attention v and output Q6_K in the last 14 single blocks and Q5_K elsewhere; the MLP output of the last 14 single blocks and the MLP input of the last 7 Q5_K; the modulation layers and the reference-image blocks at the base type.
• The exact rules and the per-tensor type map of every file are in the metrics repo (logs/convert-*.log, layouts/). AD-Q5_K is the row AD-Q5_K-u there and AD-Q4_K the row AD-Q4_K-v1.

The text encoder

Klein reads Qwen3-VL-4B-Instruct's hidden states, and the encoder's quantization moves the picture as much as the image model's. Each encoder below rendered the 48 captions on seed 42 with the BF16 image model, compared with the same renders through the BF16 encoder:

Encoder file	Size on disk	Picture LPIPS [95% CI]	Next-token KLD to BF16
text_encoders/Qwen3-VL-4B-Instruct-Q8_0.gguf	4.28 GB	0.016 [0.012, 0.020]	0.0015
text_encoders/Qwen3-VL-4B-Instruct-AD-Q6_K.gguf	3.68 GB	0.029 [0.024, 0.035]	0.0024
text_encoders/Qwen3-VL-4B-Instruct-AD-Q4_K.gguf	2.49 GB	0.070 [0.058, 0.082]	0.025
Qwen's own Qwen3VL-4B-Instruct-Q4_K_M.gguf	2.50 GB	0.088 [0.070, 0.109]	not measured; a plain Q4_K_M of ours: 0.032
Qwen's own Qwen3VL-4B-Instruct-Q8_0.gguf	4.28 GB	0.016 [0.012, 0.020]	not measured

• A Q4_K_M encoder moves the picture more (0.088) than our AD-Q4_K image model does (0.081). Use the Q8_0 encoder if it fits.
• These are one seed; the image model numbers above are two.
• The text encoder is Qwen3-VL-4B-Instruct byte for byte (the sha256 of Klein's text_encoder/ shards equals Qwen/Qwen3-VL-4B-Instruct's), so any GGUF of it works.

Running it

Klein is distilled to 16 steps at CFG 1. stable-diffusion.cpp's default schedule for it is BFL's (time shift 6.93), so no sigma list is needed.

sd-cli --diffusion-model FLUX-3-Image-Klein-AD-Q5_K.gguf \
  --llm text_encoders/Qwen3-VL-4B-Instruct-Q8_0.gguf \
  --vae vae/ae.safetensors \
  -p "$(cat caption.txt)" \
  --steps 16 --cfg-scale 1.0 --sampling-method euler -W 1024 -H 1024 -o out.png

• Use a caption, not a short prompt. Klein is trained on full scene captions written by a prompt upsampler. BFL's is a 9B model, in GGUF at AtomicChat/FLUX-3-Image-Klein-pup-GGUF with BFL's system prompts. Any capable LLM can write the caption with the t2i system prompt. A short prompt sent as typed gives a generic picture.
• stable-diffusion.cpp master-968-8cd731e or newer (FLUX 3 support, leejet/stable-diffusion.cpp#2143). We measured with 95b3090 (master-975).
• Editing: add the reference images with -r (up to 10), and write the caption from the instruction and the references (the upsampler's i2i system prompt, with its vision projector).

In ComfyUI

• Image model: our GGUF in models/unet/, loaded with ComfyUI-GGUF's Unet Loader (GGUF). Start from ComfyUI's FLUX 3 Klein template ("distilled t2i") and put this node in place of Load Diffusion Model.

• Text encoder: use qwen3vl_4b_bf16.safetensors (or qwen3vl_4b_fp8_scaled.safetensors) from Comfy-Org/Krea-2 with the core Load CLIP node, type flux3.

  • Our encoder GGUFs in text_encoders/ are for stable-diffusion.cpp. In ComfyUI, ComfyUI-GGUF loads a Qwen3-VL GGUF as plain Qwen3-4B, and the sampler stops with a shape error (2560 against 20480).

• VAE: vae/ae.safetensors with Load VAE.

• Checked on 21.10 with ComfyUI 7a28332 and ComfyUI-GGUF 6ea2651: 8 captions, seed 42, the template's 16 steps, euler, simple. Each LPIPS is against ComfyUI's own render with BFL's BF16 safetensors:

  file	LPIPS
  our BF16 GGUF	0.0000 (the same picture)
  Q8_0	0.021
  AD-Q5_K	0.031
  AD-Q4_K	0.069
  unsloth's UD-Q4_K_XL (same run)	0.052

  The renders are in the metrics repo under comfyui/.

Changes

• 2026-10-21: the four image-model files now carry general.architecture = flux and general.quantization_version = 2 in their metadata, so ComfyUI-GGUF loads them. Before, it stopped with "Unknown model architecture!".
  • Each file is 96 bytes larger. The tensors are byte for byte the same, and stable-diffusion.cpp renders the same pixels as before.
  • A file downloaded earlier keeps working in stable-diffusion.cpp.

Content filtering

The open FLUX 3 Klein pipeline has no content filter of its own. BFL's safety checks, an NSFW image classifier and optional prompt checks, live in BFL's Python inference code. stable-diffusion.cpp, ComfyUI and Atomic Chat do not run them. BFL also states that it evaluated and mitigated risks including child sexual abuse material and non-consensual intimate imagery before release.

We measured what the open pipeline draws on 45 probe requests in nine categories (adults only, no real people), upsampled by BFL's prompt upsampler and judged by an image model, two seeds, BF16 weights:

Category	Requested content drawn
Nudity (art, beach, life drawing)	8 / 10
Suggestive	7 / 10
Violence	10 / 10
Weapons	10 / 10
Drugs	9 / 10

We also removed the refusal direction from the text encoder and from the prompt upsampler (abliteration) and ran the same probes. Neither changed any category by more than one image in ten, in either direction, alone or together. The stock upsampler already writes the requested content into its captions (31 of 35 sensitive requests; the abliterated one 33, and the image model draws neither of the two extra ones). With no open component refusing, an uncensored encoder or upsampler has nothing to unlock in Klein's pictures. We publish both as models without refusals, with these numbers in their cards: FLUX-3-Image-Klein-TE-Uncensored-GGUF (the text encoder, a drop-in --llm) and FLUX-3-Image-Klein-pup-Uncensored-GGUF (the prompt upsampler). The aggregates are in the metrics repo (diagnostic/); the probe requests and their renders are not published.

The FLUX Kommunity License forbids, among other things, using the model to generate unlawful content, including child sexual abuse material and non-consensual intimate images, and violating anyone's rights of publicity. See the license, section 4.

How it was measured

• Engine: stable-diffusion.cpp at 95b3090 (master-975) on an NVIDIA H100 80 GB.
• Settings: 1024×1024, 16 steps, CFG 1.0, euler, the default schedule, --rng cpu, seeds 42 and 43.
• Prompts: 48 requests in six categories (photo, illustration, English text, Chinese text, several objects with attributes, transparent background), each upsampled to a full caption by BFL's prompt upsampler.
• Held constant: the BF16 text encoder and the autoencoder, so the image model file was the only thing that changed.
• Not measured: aesthetics or caption adherence as such, edits with reference images (a quick check with the BF16 render as its own reference showed every file at Q4 and above reproducing it within LPIPS 0.015; that does not measure real edits), other step counts, resolutions or samplers, and runtimes other than stable-diffusion.cpp.
• In the metrics repo: every render, the captions, the importance matrix, the sensitivity scan, the per-tensor layout of every file, ours and theirs, and every log.

How these files were made

1. BFL's model.safetensors and ae.safetensors at revision 3373376, checked against the Hub's sha256. vae/ae.safetensors is BFL's file, unchanged.
2. Converted with sd-cli -M convert from stable-diffusion.cpp 95b3090. The AD- files add --tensor-type-rules and --imat-in, with the exact command for each in the metrics repo.
3. The text encoders: Qwen/Qwen3-VL-4B-Instruct, converted with llama.cpp 6184e92, importance matrix from our general calibration corpus with --parse-special.

License

The image model files and the autoencoder are a modified FLUX Model under the FLUX Kommunity License; see NOTICE.md. The license allows non-commercial use, and commercial use of outputs for Qualifying Users (under US$5M annual revenue), under its terms. The text encoders are under the Apache License 2.0 (text_encoders/LICENSE). Quantized by AtomicChat; not affiliated with or endorsed by Black Forest Labs.