rshoemake/taq-qwen32b-coder-mixed-allocator-2bit

TAQ Compressed Weights — Qwen2.5-Coder-32B-Instruct, mixed_allocator (2-bit-class)

Real, byte-packed, kernel-loadable quantized weights for Qwen/Qwen2.5-Coder-32B-Instruct, produced by the project's Tail-Aware Quantization (TAQ) pipeline — the per-layer mixed-precision allocator's own choice of action (outlier/rotation_outlier at 2/3/4-bit) at a natural ~2-bit budget.

Fidelity (native family-scale evaluation, WikiText-2)

fp16 reference PPL: 11.9266 (KLD=0 by definition)

	bpw	PPL	KLD vs fp16
This checkpoint (mixed_allocator_2bit)	2.8734	20.387	0.5063
outlier_2bit (uniform baseline, same budget class)	2.8639	22.193	0.6305
rotation_outlier_2bit (uniform baseline, same budget class)	2.8734	21.597	0.6211

Mixed allocator wins on fidelity: 0.5063 KLD vs the best uniform baseline's 0.6211 at effectively the same bit budget (2.8734 bpw). No paired bootstrap significance test has been run at this scale yet.

Code generation quality (EvalPlus HumanEval+ / MBPP+, pass@1)

	HumanEval Base	HumanEval Base+Extra	MBPP Base	MBPP Base+Extra
This checkpoint (mixed_allocator_2bit)	0.8659	0.7988	0.7845	0.6466
outlier_2bit (same budget class)	0.8598	0.7988	0.7769	0.6566
rotation_outlier_2bit (same budget class)	0.8841	0.8049	0.7519	0.6391
fp16 (reference, uncompressed)	0.9024	0.8598	0.8647	0.7268

vs fp16: HumanEval Base+Extra -0.0610, MBPP Base+Extra -0.0802 (pass@1 delta, this checkpoint minus fp16 baseline).

Note on GGUF comparator: a real Unsloth GGUF Q4_K_M quant of this model was also evaluated (HumanEval Base/+Extra 0.8902/0.8293, MBPP Base/+Extra 0.8546/0.7293) — see the sibling 4-bit-tier repos (rshoemake/taq-qwen32b-coder-{outlier,rotation-outlier,mixed-allocator}-4bit) for the full row in a matched-bpw table. Not inserted directly into this table since GGUF Q4_K_M (~4.85 bpw) is not bit-matched to this checkpoint's budget.

What's in this repo

Real, byte-packed quantized weights (not a dequantized fp16 scaffold) — same format documented in full at rshoemake/taq-qwen14b-unsloth-matched: packed/*.bin per-layer files (header + fp16 codebooks + bit-packed indices + outlier side-channel), packed/manifest.json (per-layer action/bits/measured bpw), fp16_passthrough/*.safetensors for any layers the allocator left at full precision, unpack_outlier.py / unpack_rotation_outlier.py standalone unpackers.

Provenance

• Source: Qwen/Qwen2.5-Coder-32B-Instruct (fp16), packed directly from the original safetensors shards.
• Fidelity evaluation harness: run_full_matrix.py / run_tail_family_scale.py (project fractal_quant_decision, private) — native family-scale protocol (WikiText-2, passage-level KLD).
• Code-eval harness: code_eval_gen.py + evalplus.evaluate (HumanEval+/MBPP+, greedy decoding, max-new-tokens=640) — same decode/scoring config used across every arm in this comparison table so results are directly comparable.
• Packer: pack_outlier_format.py + pack_rotation_outlier.py + pack_whole_model.py.