Harsha001/flux-3-image-klein-ROCmFPX

FLUX.3 Image Klein — ROCmFPX GGUF

The diffusion transformer of black-forest-labs/flux-3-image-klein quantized to ROCmFPX formats for stable-diffusion.cpp.

These files need stable-diffusion.cpp built against the ROCmFPX fork of ggml. Stock sd.cpp, ComfyUI-GGUF and other runners do not know these types. The patch is in this repo (sdcpp-rocmfpx.patch, 29 lines).

Not an official Black Forest Labs release. Non-commercial use under the FLUX Kommunity License (see LICENSE.md and NOTICE.md).

Files and measured quality

8 prompts, 1024×1024, 16 steps, Euler, same seed per prompt. Every image is compared with the BF16 image of the same prompt and seed. PSNR (dB) and SSIM: higher is closer. LPIPS: lower is closer (perceptual distance). For scale: the same BF16 weights run in BF16 math instead of F32 already give PSNR 42.3, SSIM 0.994, LPIPS 0.004. Diffusion amplifies small weight changes over the steps, so quantized images keep the prompt and composition but differ in detail.

File	Size	PSNR	SSIM	LPIPS	Notes
flux-3-image-klein-Q8_0_ROCMFPX.gguf	5.5GB	30.4	0.969	0.025	ROCmFPX 8-bit (UE4M3 block scales)
flux-3-image-klein-Q6_0_ROCMFPX.gguf	4.4GB	26.2	0.922	0.070	ROCmFPX 6-bit
flux-3-image-klein-Q4_0_ROCMFP4.gguf	3.1GB	19.4	0.783	0.177	ROCmFP4 4-bit
flux-3-image-klein-Q4_0_ROCMFP4_FAST.gguf	2.9GB	20.2	0.794	0.175	ROCmFP4 4-bit, one scale per block (speed layout)
flux-3-image-klein-Q4_0_ROCMI4.gguf	2.9GB	19.6	0.777	0.188	signed 4-bit integers with UE4M3 scale
flux-3-image-klein-Q3_0_ROCMFPX.gguf	2.5GB	17.6	0.693	0.280	ROCmFP3 3-bit (preview format), visible quality loss
Q8_0 (stock ggml, baseline only)	5.6GB	34.6	0.984	0.011
Q4_K (stock ggml, baseline only)	3.1GB	20.7	0.816	0.154
Q4_0 (stock ggml, baseline only)	3.1GB	19.1	0.787	0.171

Honest comparison: on this model Q8_0_ROCMFPX scores below stock Q8_0 of about the same size; Q4_0_ROCMFP4 scores below stock Q4_K of about the same size. The ROCmFPX formats are built for fast AMD kernels; their speed for image models on AMD has not been measured yet.

Not published: Q2_0_ROCMFPX (LPIPS 0.72, images no longer follow the reference).

Sizes are the diffusion transformer only (BF16: 11.1 GB). Layers that sd.cpp keeps in full precision for FLUX (input/output projections, time embedding, biases, norms) stay BF16.

How these were made

1. stable-diffusion.cpp (b13a482) built on the ROCmFPX ggml (fb08d7c) with sdcpp-rocmfpx.patch.
2. Importance matrix from the BF16 model over 16 calibration prompts (different from the test prompts), 1024², 16 steps.
3. sd-cli -M convert --type <type> --imat-in imatrix.dat.
4. Quality measured on GPU with every quantized weight expanded back to F32 at load time, so the numbers measure the stored weights. The fork's own GPU kernels for these types were not used (they are wrong on NVIDIA). On AMD GPUs the native kernels may round differently; these files have not been tested on AMD hardware yet.

Run

git clone https://github.com/charlie12345/ROCmFPX
git clone https://github.com/leejet/stable-diffusion.cpp sdcpp && cd sdcpp && git checkout b13a482
git submodule update --init --recursive && git apply ../sdcpp-rocmfpx.patch
cmake -B build -DSD_USE_UPSTREAM_GGML=ON -DSD_GGML_SOURCE_DIR=$PWD/../ROCmFPX/ggml -DSD_ROCMFPX=ON -DSD_VULKAN=ON
cmake --build build -j
build/bin/sd-cli --diffusion-model flux-3-image-klein-Q4_0_ROCMFP4_FAST.gguf --llm Qwen3-VL-4B-Instruct-Q8_0.gguf \
  --vae ae.safetensors -p "a lovely cat holding a sign that says hello" --cfg-scale 1.0 --steps 16 --sampling-method euler

Text encoder: Qwen/Qwen3-VL-4B-Instruct-GGUF. VAE: ae.safetensors from the original repo.