Qwen3.8-27B-DR-4bit

Compact Bit-Packed 4-Bit (4.17 bpw, 12.52 GiB) DR Quantized Checkpoint of Qwen/Qwen3.8-27B.

• Compact Packed Checkpoint (12.52 GiB): Grawnch/Qwen3.8-27B-DR-4bit-Packed — stores bit-packed integer weights (qweight) and FP16 group scales (scales) in a single 12.52 GiB .safetensors file with on-the-fly low-VRAM inference (DRLinear).
• Pre-Unpacked Transformers Evaluation Checkpoint: Grawnch/Qwen3.8-27B-DR-4bit — pre-reconstructed bfloat16 weights for 1-line AutoModelForCausalLM.from_pretrained benchmarking.

Quickstart (Low-VRAM Inference)

from huggingface_hub import hf_hub_download
from dr_loader import load_dr_packed

ckpt = hf_hub_download(repo_id="Grawnch/Qwen3.8-27B-DR-4bit-Packed", filename="model_packed.safetensors")
# low_vram=True keeps weights compressed in GPU VRAM (12.52 GiB); low_vram=False unpacks into standard nn.Linear
model, tokenizer = load_dr_packed(ckpt, device="cuda:0", low_vram=True)

Head-to-Head Benchmark vs. Standard 4-Bit Baselines (Qwen/Qwen3.8-27B)

Metric	16-Bit Baseline	Standard 4-Bit (RTN / Uncalibrated)	Standard 4-Bit (GPTQ)	DR (4-Bit)
Effective Bit-Width / Packed Size	16.0 bpw (51.4 GiB)	4.17 bpw (12.52 GiB)	4.17 bpw (12.52 GiB)	4.17 bpw (12.52 GiB)
Total Layer-Wise Output MSE	0.0	36,932	5,780	2,889 (2.00x)
Output Neurons Won vs. GPTQ	—	—	—	3,440,765 / 3,899,392 (88.24%)
In-Domain Perplexity (PPL)	2.518	2.576	2.548	2.509
Full-Vocab Forward KLD	0.00000	0.05692	0.04724	0.01827
Top-1 Token Agreement	100.00%	91.52%	92.69%	95.32%
Top-5 Token Agreement	100.00%	99.71%	100.00%	100.00%
Logit Cosine Similarity	1.00000	0.99015	0.99188	0.99474
EvalPlus HumanEval+ (164, 80x tests)	90.85% (149/164)	—	85.98% (141/164)	90.85% (149/164)
EvalPlus MBPP+ (378, 35x tests)	84.66% (320/378)	—	79.89% (302/378)	83.07% (314/378)