WiktorMatuszek/MOSS-Image-SFT-INT8-WO-Core

MOSS-Image-SFT — INT8-WO-Core

Release status

Experimental checkpoint — further testing recommended. Small original 50-step and/or separate experimental 10-step sample results are available. Broader GPU validation and independent Polish glyph and visual-quality assessment remain outstanding.

Benchmark interpretation: The image gallery and measurements are reference examples for visual inspection, not a required reproducibility package or guarantee of quality, speed, or VRAM usage on another setup.

Original MOSS-Image-SFT DiT with genuinely packed INT8 weight-only weights. The model is quantized from the original SFT checkpoint; there is no extra training, LoRA, distillation or architecture change.

Property	This checkpoint
Quantization	Signed INT8 weights plus scales
Coverage	228 / 269 transformer Linear projections (precision-sensitive edge layers retained in BF16)
Stored DiT weights	5.470 GiB, 42.2% smaller than original 9.467 GiB BF16 DiT
Checkpoint files	Three TorchAO/PyTorch .bin weight shards; not generic safetensors
Upstream revision	OpenMOSS-Team/MOSS-Image-SFT, fc5de0d56fc910cbca6f6625a70f5f582b26deb1
Integrity	All 463 original state keys matched; shards reloaded and SHA-256 verified

This repo contains the transformer only. The original Qwen text encoder, tokenizer, FP32 VAE, scheduler and pipeline code are loaded from the pinned upstream revision as needed. The stored DiT size is not total RAM/VRAM or whole-pipeline model size.

Evidence and measured limits

Weights: all 228 quantized Linear weight matrices compared with the original BF16 tensors: parameter-weighted MSE 2.49e-07, SQNR 39.47 dB, cosine 0.999944. These are weight-space errors, not generated-image correctness. Detailed per-layer measurements, quantization receipt, checkpoint integrity.

Original SFT50 image study: 5 matched 1024px pairs, seed 42, 50 steps, CFG 3. SSIM 0.8950, PSNR 23.30 dB, LPIPS (SqueezeNet) 0.0573 against original BF16. Smoke-sized, not the full image or Polish typography benchmark. Paired-image metrics.

Separate Turbo10 experiment: 16 paired 1024px cases, seed 42, DPM++2M10, CFG 3, CMP 170HX / SM80. Median 12.074 s/image, 10.344 GiB CUDA reserved; SSIM 0.9097, learned LPIPS 0.053268 vs BF16. The BF16 control required 11.029 s on that same path, so this quantization did not provide a speedup there. Neither these times nor their pixel-quality scores should be generalized across GPUs or equated with SFT50. Raw results.

Use with original SFT inference

Install a CUDA-compatible PyTorch for your GPU, followed by the repository requirements. The conversion environment was PyTorch 2.7.0+cu128 / TorchAO 0.12.0+cu128; compatibility with packed tensor subclasses and native kernels is device dependent.

hf download WiktorMatuszek/MOSS-Image-SFT-INT8-WO-Core --local-dir ./moss-quant
cd moss-quant
python -m pip install -r requirements.txt

# Check all real saved shards before using the model
python generate.py --verify-only

# Original MOSS-SFT: 50 denoising steps, CFG 3, 1024px
python generate.py --prompt-file example_layout.json --output image.png

The launcher downloads the unchanged missing upstream components, not duplicate original BF16 DiT weights. Optional Qwen FP8 encoder / Flux2 VAE FP16 can be selected with --encoder-repo and --vae-repo; that combination needs its own end-to-end quality validation.

Real image examples

Paired original 50-step SFT output: same prompt and seed, BF16 on the left and this exact saved checkpoint on the right.

A different 10-step experimental sampler, same seed and prompts (BF16 / INT8 Core / FP8 Core):

Other original generated images and full comparison.

Files and limitations

transformer/ contains the quantized checkpoint; generate.py, run_quantized_sft.py, compact_components.py contain the strict pinned loader; QUANTIZATION_RECEIPT.json and COMPLETE.json record checksums. Frozen prompt suites are archived once in the research dataset, not copied into this model repo.

Pixel similarity does not prove correct Polish spelling, photorealism, aesthetic quality or native-kernel support. Test speed and GPU memory on your intended stack. The full cross-format gallery, including explicitly missing measurements, is maintained once in the benchmark dataset.

Attribution: independently quantized from OpenMOSS-Team/MOSS-Image-SFT at fc5de0d56fc910cbca6f6625a70f5f582b26deb1; original Apache-2.0 license in LICENSE. Not an official OpenMOSS-Team release.