BennyDaBall/MOSS-Image-SFT-NVFP4

MOSS-Image SFT NVFP4 for ComfyUI

Follow me on X @BennyDaBall_OG !

This is an independent NVFP4 quantization of MOSS-Image SFT. The OpenMOSS team created the original model. I converted Comfy-Org's official bf16 checkpoint and packaged the quantized weights and the validation here. This is not a new base model or an official OpenMOSS release.

One file. 4.49 GB. 2.5x faster than bf16 on an RTX 5090, and faster than the official int8 too.

As far as I can tell, this is the first real NVFP4 build of MOSS-Image you can drop into stock ComfyUI. No custom nodes, no runtime patch, no GGUF loader. Load Diffusion Model, pick the file, hit Run.

Model Metadata

Key	Value
License	Apache-2.0 (same as the source weights)
Base Model	OpenMOSS-Team/MOSS-Image-SFT (via Comfy-Org/MOSS-Image bf16)
Architecture	~5B single-stream DiT, 42 blocks, Qwen3.5-9B text encoder (unchanged)
Format	ComfyUI native NVFP4 + int8 ConvRot + bf16, per-layer comfy_quant metadata
File	moss_image_sft_nvfp4.safetensors, 4.49 GB
Tested on	RTX 5090, ComfyUI 0.42.1, comfy-kitchen 0.2.39, PyTorch 2.14 cu130
sha256	b478cb628a7cbfccd9d04c107b7c649a1b0facbf631a91c59ef2ab9df2ec5ef6

---

What is MOSS-Image, and why should you care?

MOSS-Image is OpenMOSS's text-to-image model, and its party trick is text. Signs, posters, labels, menus, handwritten notes, in English, Chinese, Korean, Arabic and more, spelled correctly a lot more often than you'd expect from a diffusion model. It also takes layout boxes, so you can tell it where the car goes, where the headline goes and what the headline says.

The catch: the bf16 model is 10.16 GB and takes 27 seconds per 1920x1088 image on a 5090. This build gets that down to 4.49 GB and 11 seconds while keeping the text sharp.

Every image above was rendered with this NVFP4 file at 1920x1088, official template settings, best of 2 seeds, no upscaling, no inpainting, no touch-ups. The WILD cover was placed with MOSS-Image's layout boxes (Create Bounding Boxes into MOSS-Image Layout Prompt). Full-resolution PNGs are in images/hero.

---

The Numbers

Same machine, same template (er_sde, simple, 25 steps, cfg 2.5, the template's negative prompt). Speed is ComfyUI's own execution time per image, median of 5 warm runs, a fresh seed every run so nothing gets cached. Peak VRAM is the whole GPU (text encoder, VAE and my ~3.3 GB desktop included).

Model	File	s/img 1920x1088	s/img 1024x1024	Peak VRAM
bf16 (Comfy-Org)	10.16 GB	27.04	12.98	22.1 GB
int8 ConvRot (Comfy-Org)	5.41 GB	11.79	5.37	17.6 GB
NVFP4 (this repo)	4.49 GB	10.96	4.92	16.2 GB

So: 2.5x faster than bf16, 7-8% faster than int8, 56% smaller than bf16, 17% smaller than int8.

How close is it to the original? Same prompt, same seed, measured against bf16 on 6 prompts x 2 seeds (an English sign, the official Chinese fridge-note prompt, an English/Arabic/Korean sign, a portrait, a sky gradient and a busy night market). Lower LPIPS = closer to bf16.

Model	LPIPS	SSIM	PSNR	Color shift (R, G, B)
int8 ConvRot (Comfy-Org)	0.133	0.882	23.8 dB	+4, +2, +1
NVFP4 (this repo)	0.170	0.844	21.5 dB	+4, +1, 0
Plain NVFP4, no GPTQ, every block 4-bit	0.362	0.682	15.4 dB	+27, +21, +6

That last row is what you get from the "just quantize every block to FP4" recipe, and it is why this repo exists. More on that below.

Text check. Every string in those text prompts was read back letter by letter by a reader that didn't know which image came from which model (16 strings per model):

Model	Exact strings	Letter errors
NVFP4 (this repo)	16/16	0
int8 ConvRot	15/16	2 (Arabic)
bf16	14/16	3

Yes, the 4-bit file beat the original on this one. No, I'm not going to pretend that means it's smarter than bf16. It's a small sample, so read it as "no text regression", which is the part that matters.

---

Same prompts, same seeds

bf16 on the left, this NVFP4 on the right. Same prompt, same seed, same everything.

At 2048x1152, 100% crop: no mesh, streaking or banding. Same prompt, same seed.

Honest take: same-seed images can land a little differently in layout (int8's do too), but color, contrast, detail and text hold up. Comfy-Org's int8 is still a hair closer to bf16 pixel for pixel. This one trades a sliver of that for speed and size.

---

Quick Start

1. Download moss_image_sft_nvfp4.safetensors into ComfyUI/models/diffusion_models/.
2. Open the official MOSS-Image template (image_moss_image_t2i).
3. In Load Diffusion Model, pick moss_image_sft_nvfp4.safetensors and leave weight_dtype on default.
4. Text encoder and VAE stay the same as the template (for example qwen3.5_9b_w4a8_gptq with CLIP type moss_image, and flux2-vae_small).
5. Run.

Requirements:

• ComfyUI with MOSS-Image support (0.41.3 or newer; I tested 0.42.1).
• An RTX 50-series / Blackwell GPU for native FP4. On older cards ComfyUI falls back to dequantizing the FP4 layers. It works, it's just slow. Grab Comfy-Org's int8 file instead if that's you.

Verified settings:

UNET: moss_image_sft_nvfp4.safetensors (weight_dtype default)
Text Encoder: qwen3.5_9b_w4a8_gptq.safetensors (type moss_image)
VAE: flux2-vae_small.safetensors
Steps: 25
CFG: 2.5
Sampler: er_sde
Scheduler: simple
Latent: EmptyFlux2LatentImage, 1-2 megapixels

---

Under the Hood: what's actually in the file?

Layers	Format	Why
qkv, o, ffn_up_gate in blocks 0-23 (72 layers)	NVFP4, GPTQ-rounded	These blocks are FP4-safe, so they get the fast Blackwell FP4 tensor cores
qkv, o, ffn_up_gate in blocks 24-41	int8 ConvRot	The late blocks are where 4-bit hurts, measured below
ffn_down in all 42 blocks	int8 ConvRot	Speed, not quality: see "the SiTU problem"
Embedders, 2-block text refiner, time embedding, both modulations, final layer, norms (17 layers)	bf16	Tiny, run once per step, not worth a single bit of risk

• NVFP4 layers are ComfyUI's native format: 4-bit e2m1 weights, one e4m3 scale per 16 weights, one fp32 scale per tensor, activations quantized on the fly (W4A4). Nothing extra stored, no calibration scales.
• The int8 ConvRot layers are copied byte for byte from Comfy-Org's moss_image_sft_int8_convrot.safetensors. They did good work, I kept it.
• Every block carries the same comfy_kitchen_int8 attention setting as Comfy-Org's int8 file.

How we got here (the short version of a long two days)

1. The obvious recipe failed. Every block linear in FP4, everything else bf16, round-to-nearest. 3.33 GB, very tempting. It came out slower than int8 and gave every image a warm, oversaturated, golden-hour-filter look nobody asked for.
2. The SiTU problem. MOSS uses a soft-capped gated activation ("SiTU") in front of each ffn_down. Comfy-Org's int8 kernel fuses it into its quantize step. The FP4 path runs it as a pile of separate little kernels, which ate about 30% of every forward pass. The FP4 matrix multiplies themselves are great (cutlass3x_sm120 ... block_scaled_ue4m3xe2m1, ~40% faster than int8's), so the fix is simple: ffn_down stays int8, everything else gets to sprint.
3. Finding the tint. I replayed the exact bf16 denoiser inputs through the quantized model and measured where the error came from. Almost all of it was a global color bias, almost all of it from rounding the 4-bit weights, and almost all of it in blocks 28-41.
4. GPTQ fixed the weights. Instead of rounding each weight on its own, GPTQ rounds each layer so its output stays as close to bf16 as possible on real MOSS activations (10 calibration prompts, both CFG branches, 13 of the 25 steps, none of them reused in the tests). Each 16-weight block scale is picked by trying all 126 FP8 codes. Result: per-layer output error down 4-6x, and the tint gone.
5. A block-by-block sweep picked the int8 tail. With GPTQ weights, I protected one block at a time and measured the gain on held-out prompts. Sensitivity climbs steadily with depth (block 41 matters most, then 40, 39...), and blocks 0-26 are basically free to run in FP4. So the last 18 blocks keep int8, the first 24 run FP4.
6. Checked it like I meant it. File read back against the converter's own math, loaded through the stock loader, speed raced, A/B'd against bf16, text read back letter by letter.

---

Verification & Proof

• Speed: 5 warm runs per model, fresh seeds, ComfyUI execution timestamps.
• Quality: LPIPS / SSIM / PSNR / color shift against same-seed bf16 at 1024x1024, plus a 2048x1152 check (LPIPS 0.111 int8, 0.195 this file; no mesh, streaking or banding at 100%).
• Text: blind letter-by-letter read, scored by edit distance over letters and digits in any script.
• Prompts used for tuning and prompts used for testing never overlap.

---

Limitations

• Mixed precision, not FP4 everywhere. The FP4 share is the part that helps on Blackwell without hurting the picture.
• Not tested with LoRAs. Heads up: ComfyUI requantizes FP4 layers when it patches a LoRA onto them.
• My numbers are from one RTX 5090. Other Blackwell cards should see the same ranking, not necessarily the same seconds.
• Diffusion is still diffusion. A good file won't save a bad seed.

---

Acknowledgements

• OpenMOSS for MOSS-Image (OpenMOSS-Team/MOSS-Image-SFT), Apache-2.0.
• Comfy-Org for native ComfyUI support, the bf16 repack and the int8 ConvRot layers (Comfy-Org/MOSS-Image), Apache-2.0.
• comfy-kitchen for the NVFP4 runtime, and the GPTQ authors (Frantar et al.) for the rounding trick that saved this build.
• My local power utility provider, for once again sustaining the research grid.

This repo is released under Apache-2.0, same as the source weights. Not affiliated with OpenMOSS or Comfy-Org.

Quantized & tested locally with care by BennyDaBall.

Follow me on X @BennyDaBall_OG !