Add 21.7GB mixed int8 + w4a8 quant (24GB VRAM target)
#23
by cicalooo - opened
Mixed int8_tensorwise + asym_w4a8_int8 (group 4, ConvRot-256) quant of10Eros_Max_FAST-H3_ref_beta6_unstable_pruned.safetensors, sized at 21.71GB
(21,710,640,715 bytes) to fit 24GB VRAM with headroom for VAE/text encoders.
Recipe (quantized from the bf16 pruned file in this repo):
- 131x int8: all attn qkv/out, edge MLP (blocks 0-5, 44-49), heads, token_refiner attn
- 80x w4a8-g4: middle MLP (blocks 6-43), token_refiner MLP
- Kept BF16: gates, norms, adaln, patch projs
Checks: contiguous offsets, finite/positive scales, kept tensors byte-identical;
spot dequant rel_L2 int8 ~0.01, w4a8 ~0.054.
Load with stock ComfyUI UNETLoader, weight_dtype=default.
By cicalooo (also on my gated repo
cicalooo/10Eros_Max_FAST-H3_ref_beta6-mix_w4a8g4-int8).
Merge or close whenever suits β no rush.
cicalooo changed pull request status to open
TenStrip changed pull request status to merged