Harsha001/MOSS-Image-SFT-ROCmFPX

MOSS-Image-SFT — ROCmFPX GGUF

The 5B diffusion transformer of OpenMOSS-Team/MOSS-Image-SFT for stable-diffusion.cpp, in Q8_0_ROCMFPX, the 8-bit ROCmFPX format for AMD GPUs.

This file needs stable-diffusion.cpp built against the ROCmFPX fork of ggml. Stock sd.cpp, ComfyUI-GGUF and other runners do not know its type. The patch is in this repo (sdcpp-rocmfpx.patch).

Not on AMD with ROCmFPX? Use a stock Q8_0, for example from leejet/MOSS-Image-SFT-GGUF. In the test below a stock Q8_0 made the same way as this file is closer to BF16 and only 0.15 GB larger. This file is for trying the ROCmFPX kernels on AMD.

Not an official release of OpenMOSS. All credit for the model goes to the OpenMOSS team.

File

File	Size
MOSS-Image-SFT-Q8_0_ROCMFPX.gguf	5.56 GB	the diffusion transformer only; text encoder and VAE come from the original repo

Only the 252 main projections are quantized (attention q/k/v/o, ffn_up_gate, ffn_down in every block). Embedders, norms, modulation, time conditioning and the output layer stay BF16, as in the reference MOSS GGUFs.

Measured quality

8 prompts (photos, illustration, two with written text), 1024×1024, Euler, 30 steps, CFG 3.0, flow shift 5, one fixed seed per prompt. Every image is compared with the BF16 image of the same prompt and seed. PSNR (dB) and SSIM: higher is closer. LPIPS: lower is closer (perceptual distance). Diffusion amplifies small weight changes over the steps, so a quantized model can give a different take of the same prompt (for example other words on a sign) without any loss of quality.

Weights	Size	PSNR	SSIM	LPIPS
BF16 loaded directly (floor of this test)	10.2 GB	33.4	0.974	0.022	original
Q8_0_ROCMFPX	5.56 GB	26.6	0.916	0.074	this repo
Q8_0 (stock ggml, same tensors)	5.71 GB	28.9	0.949	0.043	baseline, not published here
Q6_0_ROCMFPX	4.52 GB	22.1	0.850	0.126	not published
Q4_0_ROCMFP4	3.33 GB	16.5	0.665	0.294	not published
Q4_0_ROCMFP4_FAST	3.18 GB	15.5	0.655	0.323	not published
Q4_0_ROCMI4	3.18 GB	16.1	0.639	0.326	not published

The 6- and 4-bit files change the images too much (mostly different compositions, and text that changes) to be worth publishing for a model this small.

How this was made

1. stable-diffusion.cpp (b13a482, which supports MOSS-Image) built on the ROCmFPX ggml (fb08d7c) with sdcpp-rocmfpx.patch.
2. Importance matrix from the BF16 transformer over 16 calibration prompts (different from the test prompts), same settings.
3. sd-cli -M convert --tensor-type-rules '^layers\..*\.proj\.(q|k|v|o|ffn_up_gate|ffn_down)\.weight$=q8_0_rocmfpx' --imat-in imatrix.dat
4. Quality measured on an NVIDIA GPU with every transformer weight expanded back to full precision and rendered with sd.cpp, so the numbers measure the stored weights. The fork's own GPU kernels for these types were not used (they are wrong on NVIDIA). Text encoder for all rows: the original BF16 Qwen3.5 9B from the MOSS repo. AMD GPU kernels compute the same math in a different order, so results on an AMD GPU can differ slightly. This file has not been tested on AMD hardware yet.

Run

git clone https://github.com/charlie12345/ROCmFPX
git clone https://github.com/leejet/stable-diffusion.cpp sdcpp && cd sdcpp && git checkout b13a482
git submodule update --init --recursive && git apply ../sdcpp-rocmfpx.patch
cmake -B build -DSD_USE_UPSTREAM_GGML=ON -DSD_GGML_SOURCE_DIR=$PWD/../ROCmFPX/ggml -DSD_ROCMFPX=ON -DSD_VULKAN=ON
cmake --build build -j
build/bin/sd-cli --diffusion-model MOSS-Image-SFT-Q8_0_ROCMFPX.gguf \
  --llm Qwen3.5-9B-Q8_0.gguf --vae flux2_ae.safetensors --tokenizer tokenizer.json \
  --cfg-scale 3.0 --steps 30 -W 1024 -H 1024 -p "a lovely cat holding a sign that says 'ROCmFPX'" -o cat.png

Text encoder, VAE and tokenizer: see sd.cpp's MOSS-Image guide (Qwen3.5 9B GGUF, the FLUX.2 VAE, and tokenizer.json from the MOSS repo; the external tokenizer is required). Use image sizes divisible by 16.

License

Apache-2.0, same as the base model (LICENSE).