Qwen3.8-27B-DR-3bit

Compact Bit-Packed 3-Bit (3.17 bpw, 9.69 GiB) DR Quantized Checkpoint of Qwen/Qwen3.8-27B.

• Compact Packed Checkpoint (9.69 GiB): Grawnch/Qwen3.8-27B-DR-3bit-Packed — stores bit-packed integer weights (qweight) and FP16 group scales (scales) in a single 9.69 GiB .safetensors file with on-the-fly low-VRAM inference (DRLinear).
• Pre-Unpacked Transformers Evaluation Checkpoint: Grawnch/Qwen3.8-27B-DR-3bit — pre-reconstructed bfloat16 weights for 1-line AutoModelForCausalLM.from_pretrained benchmarking.

Quickstart (Low-VRAM Inference)

from huggingface_hub import hf_hub_download
from dr_loader import load_dr_packed

ckpt = hf_hub_download(repo_id="Grawnch/Qwen3.8-27B-DR-3bit-Packed", filename="model_packed.safetensors")
# low_vram=True keeps weights compressed in GPU VRAM (9.69 GiB); low_vram=False unpacks into standard nn.Linear
model, tokenizer = load_dr_packed(ckpt, device="cuda:0", low_vram=True)

Head-to-Head Benchmark vs. Standard 3-Bit Baselines (Qwen/Qwen3.8-27B)

Metric	16-Bit Baseline	Standard 3-Bit (RTN / Uncalibrated)	Standard 3-Bit (GPTQ)	DR (3-Bit)
Effective Bit-Width / Packed Size	16.0 bpw (51.4 GiB)	3.17 bpw (9.69 GiB)	3.17 bpw (9.69 GiB)	3.17 bpw (9.69 GiB)
Total Layer-Wise Output MSE	0.0	158,315	23,306	11,394 (2.05x)
Output Neurons Won vs. GPTQ	—	—	—	3,767,599 / 3,899,392 (96.62%)
In-Domain Perplexity (PPL)	2.518	2.910	2.780	2.647
Full-Vocab Forward KLD	0.00000	0.25128	0.14575	0.07684
Top-1 Token Agreement	100.00%	80.70%	83.63%	89.49%
Top-5 Token Agreement	100.00%	97.66%	98.54%	100.00%
Logit Cosine Similarity	1.00000	0.96144	0.97468	0.98191
EvalPlus HumanEval+ (164, 80x tests)	90.85% (149/164)	—	78.66% (129/164)	88.41% (145/164)
EvalPlus MBPP+ (378, 35x tests)	84.66% (320/378)	—	74.34% (281/378)	81.22% (307/378)