AtomicChat/FLUX-3-Image-Klein-pup-GGUF

FLUX 3 Image Klein Prompt Upsampler, GGUF

BFL's prompt upsampler for FLUX 3 Image Klein as GGUF for llama.cpp, with our own importance matrix. Every number below was measured on one stand, ours and unsloth's files alike; the logs are public.

---

FLUX 3 Image Klein is meant to read full scene captions, not short prompts. BFL's prompt upsampler is a 9B fine-tune of Qwen3.5-9B that turns a short request into such a caption: about 200 words on medium, scene, lighting, camera and every subject. This repo has it as GGUF. The image model itself is in AtomicChat/FLUX-3-Image-Klein-GGUF. The same upsampler without refusals, and a LoRA for these files, is in AtomicChat/FLUX-3-Image-Klein-pup-Uncensored-GGUF.

Pick a file

KLD is the mean KL divergence of each file's next-token distribution from the BF16 model, on a 30-language neutral text set (87 chunks of 4096 tokens). Top-1 is how often the file picks the same next token as BF16. Lower KLD and higher top-1 are closer to the original.

File	Size on disk	KLD	p99 KLD	Top-1
Q8_0	9.53 GB	0.0011	0.008	98.4%
AD-Q5_K_M	6.57 GB	0.0065	0.054	96.0%
AD-IQ4_XS	5.51 GB	0.0265	0.243	92.3%
AD-IQ3_S	4.68 GB	0.0476	0.375	89.3%

• Q8_0 if it fits.
• AD-Q5_K_M at 6.6 GB sits between unsloth's UD-Q4_K_XL and Q6_K in size and is closer to the original than UD-Q4_K_XL.
• AD-IQ4_XS is the pick at about 5.5 GB. It is smaller than unsloth's Q4_K_M and closer to the original on every measure.
• AD-IQ3_S if memory is tight. Captions get noticeably plainer below this size, so we did not publish anything smaller.
• mmproj-FLUX-3-Image-Klein-pup-F16.gguf is the vision projector, needed only for edits with reference images.

Against unsloth's files, same stand

File	Size on disk	KLD	p99 KLD	Top-1
unsloth Q8_0	9.53 GB	0.0011	0.008	98.4% (the same weights as our Q8_0)
unsloth Q6_K	7.36 GB	0.0033	0.024	97.1%
unsloth UD-Q4_K_XL	5.97 GB	0.0154	0.126	93.9%
unsloth Q4_K_M	5.63 GB	0.0300	0.265	91.8%

Unsloth's Q6_K beats our 7.6 GB candidate, so we did not publish one at that size: take theirs there.

Running it

The upsampler is a chat model with a fixed system prompt. system_prompts.json here is BFL's file, unchanged: t2i for text to image, i2i for edits, *_bbox for layouts with bounding boxes.

llama-server -m FLUX-3-Image-Klein-pup-AD-IQ4_XS.gguf --jinja -ngl 99 -c 8192

Send the t2i system prompt and the short request as the user message, with thinking off ("chat_template_kwargs": {"enable_thinking": false}), greedy decoding and a presence penalty of 1.5 (BFL's settings for text to image; 0 for edits). The answer is the caption: pass it to the image model as the prompt.

Any other capable LLM can do the job with the same system prompt; this one was tuned by BFL against Klein's renders.

How it was measured and made

• BF16: converted from black-forest-labs/flux-3-image-klein-pup at 3f3edb5 with llama.cpp 6184e92.
• Importance matrix: our calibration corpus for the Qwen3.5 / 3.8 line (the same chat template and 248k vocabulary), --parse-special, on the BF16.
• Layout: our dense-hybrid ladder, the one behind AtomicChat/Qwen3.8-27B-GGUF: attention K/V at Q8_0, the output head high (Q6_K in AD-IQ4_XS) and the embedding table at the base type, the MLP of the first two and last six blocks one step up.
• KLD: llama-perplexity --kl-divergence against the BF16's logits on the same text, the same build and GPU (NVIDIA H100) for every file, ours and unsloth's.
• Not measured: caption quality as such, or what each file does to the final image.

The upsampler is under the Apache License 2.0, by BFL on top of Qwen3.5-9B by the Qwen team. Quantized by AtomicChat; not affiliated with or endorsed by Black Forest Labs.