rshoemake/taq-qwen32b-mixed-allocator-2bit

TAQ Compressed Weights — Qwen2.5-32B-Instruct, mixed_allocator (2-bit-class)

Real, byte-packed, kernel-loadable quantized weights for Qwen/Qwen2.5-32B-Instruct, produced by the project's Tail-Aware Quantization (TAQ) pipeline — the per-layer mixed-precision allocator's own choice of action (outlier/rotation_outlier at 2/3/4-bit) at a natural ~2-bit budget.

Fidelity (native family-scale evaluation, WikiText-2)

fp16 reference PPL: 5.4541 (KLD=0 by definition)

	bpw	PPL	KLD vs fp16
This checkpoint (mixed_allocator_2bit)	2.8734	15.563	1.2452
outlier_2bit (uniform baseline, same budget class)	2.8639	21.093	1.5367
rotation_outlier_2bit (uniform baseline, same budget class)	2.8734	18.101	1.4163

Mixed allocator wins: 1.2452 KLD vs the best uniform baseline's 1.4163 at effectively the same bit budget (2.8734 bpw). No paired bootstrap significance test has been run at this scale yet — treat this as a directional result pending that analysis.

What's in this repo

Real, byte-packed quantized weights (not a dequantized fp16 scaffold) — same format documented in full at rshoemake/taq-qwen14b-unsloth-matched: packed/*.bin per-layer files (header + fp16 codebooks + bit-packed indices + outlier side-channel), packed/manifest.json (per-layer action/bits/measured bpw), fp16_passthrough/*.safetensors for any layers the allocator left at full precision, unpack_outlier.py / unpack_rotation_outlier.py standalone unpackers.

Provenance

• Source: Qwen/Qwen2.5-32B-Instruct (fp16), packed directly from the original safetensors shards.
• Evaluation harness: run_full_matrix.py / run_tail_family_scale.py (project fractal_quant_decision, private) — native family-scale protocol (WikiText-2, passage-level KLD).
• Packer: pack_outlier_format.py + pack_rotation_outlier.py + pack_whole_model.py.
• No vendor-matched (Unsloth bpw-matched) comparison has been run at this scale yet — this card reports fidelity against the fp16 baseline and against this project's own uniform baselines only.