Add 21.7GB mixed int8 + w4a8 quant (24GB VRAM target)

#23

Mixed int8_tensorwise + asym_w4a8_int8 (group 4, ConvRot-256) quant of
10Eros_Max_FAST-H3_ref_beta6_unstable_pruned.safetensors, sized at 21.71GB
(21,710,640,715 bytes) to fit 24GB VRAM with headroom for VAE/text encoders.

Recipe (quantized from the bf16 pruned file in this repo):

  • 131x int8: all attn qkv/out, edge MLP (blocks 0-5, 44-49), heads, token_refiner attn
  • 80x w4a8-g4: middle MLP (blocks 6-43), token_refiner MLP
  • Kept BF16: gates, norms, adaln, patch projs

Checks: contiguous offsets, finite/positive scales, kept tensors byte-identical;
spot dequant rel_L2 int8 ~0.01, w4a8 ~0.054.

Load with stock ComfyUI UNETLoader, weight_dtype=default.

By cicalooo (also on my gated repo
cicalooo/10Eros_Max_FAST-H3_ref_beta6-mix_w4a8g4-int8).
Merge or close whenever suits β€” no rush.

cicalooo changed pull request status to open
TenStrip changed pull request status to merged

Sign up or log in to comment