jajmangold/FLUX.1-dev-fni8

FLUX.1-dev — fni8 (int8 dp4a)

Quantization of black-forest-labs/FLUX.1-dev to the .fni8 resident format (~17.0 GB) for the fni8 W8A8 DP4A kernels on NVIDIA Volta (sm_70) — Tesla V100 / CMP 100-210.

• Weights: int8 per-row (W8A8), fp32 scales, stored in the resident dp4a VRAM layout (loads with no dequant/repack).
• Source dtype: bfloat16 (raw tensors kept fp16, upcast to fp32 only on overflow)
• Why dp4a: sm_70 has no int8 tensor cores; the contraction runs on the __dp4a CUDA-core intrinsic. On the CMP-100-210 fleet (firmware-gimped fp16 tensor cores) dp4a is the fast path, not a compromise.
• Runtimes: fni8-serve (LLMs) / ComfyUI-fni8 (diffusion DiTs).

This is a derivative quantization; its license follows the parent model above.