sorryhyun/MOSS-Image-Base-int8

MOSS-Image Base DiT, INT8 (GPTQ, ConvRot)

The DiT of MOSS-Image Base in one 5.04 GiB file, with the 252 block linears in int8. Generation at 1024² peaks at 6.3 GiB against 10.7 GiB in bf16, and denoises in 11–15 s against 21.6 s.

File	moss_image_base_int8_gptq.safetensors (5.04 GiB: 4.43 GiB int8, 0.60 GiB bf16)
Quantized	layers.N.proj.{q,k,v,o,ffn_up_gate,ffn_down}: each weight rotated per 256-column group by a Hadamard matrix (ConvRot), int8 with one fp32 weight_scale per output row
Kept in bf16	everything else: embedders, modulation, norms, head
Format	ComfyUI's int8_tensorwise + convrot layout. Loading it in ComfyUI has not been tested
Base only	made from MOSS-Image Base; SFT has no file

How it was made

Offline GPTQ (block 128, damping 0.01, activation-ordered), one block at a time, each quantized block feeding the next. Calibration: 233 images, t2i and i2i together, 466 samples (with its caption, and with the empty prompt skipping block 22, as CFG does) at σ drawn from the 50-step, shift-5 inference schedule.

The header's base_fingerprint ties the file to that DiT, and int8_id identifies its int8 tensors.

Measured

RTX 5070 Ti 16 GB. Generation, one render per arm (ER-SDE 50 steps + Spectrum, CFG 3, seed 7, compiled, all blocks resident):

job	arm	denoise	peak
t2i 1024²	bf16	21.6 s	10.7 GiB
t2i 1024²	int8 GPTQ	11.3 s	6.3 GiB
t2i 1024²	int8 GPTQ, fp32 stream	15.1 s	7.8 GiB
i2i 1216×864, full-size reference	bf16	29.1 s	13.0 GiB
i2i 〃	int8 GPTQ	27.7 s	8.6 GiB
i2i 〃	int8 GPTQ, fp32 stream	27.7 s	11.2 GiB

Read int8 generation as 11–15 s: round-to-nearest int8 runs the same kernels and took 15.0 s, so that gap is run-to-run noise.

Prediction error ‖v − v_fp32‖ / ‖v_fp32‖ in %, on images outside calibration (t2i six images; i2i 12 pairs, centre-cropped to 48×48):

	σ	bf16	int8 GPTQ	GPTQ, fp32 stream
t2i	0.1	4.81	5.93	3.85
t2i	0.5	1.93	2.43	1.71
t2i	0.9	2.08	2.30	1.34
i2i	0.1	3.61	4.50	3.52
i2i	0.5	2.30	2.84	2.17
i2i	0.9	2.05	2.24	1.19

Alone, this file is less accurate than bf16. On an fp32 residual stream it sits below bf16 at every σ tested.

Not measured

• A complete LoRA trained on this base. Training on it has stopped at 60 steps, on one set.
• A multi-seed render A/B against bf16.
• i2i error beyond sanitize crops.

Refused or unsupported

Written for anima_lora, whose int8 entry points refuse block swap (the offloader moves .weight, not the int8 copies) and LoHa (it builds W + ΔW against the base weight). LoRA and LoKr run, and a LoRA trained on bf16 loads on this base.

License: Apache-2.0, as MOSS-Image.