Qwen3.8-27B-DR-2bit

Compact Bit-Packed 2-Bit (2.17 bpw, 6.86 GiB) DR Quantized Checkpoint of Qwen/Qwen3.8-27B.

• Compact Packed Checkpoint (6.86 GiB): Grawnch/Qwen3.8-27B-DR-2bit-Packed — stores bit-packed integer weights (qweight) and FP16 group scales (scales) in a single 6.86 GiB .safetensors file with on-the-fly low-VRAM inference (DRLinear).
• Pre-Unpacked Transformers Evaluation Checkpoint: Grawnch/Qwen3.8-27B-DR-2bit — pre-reconstructed bfloat16 weights for 1-line AutoModelForCausalLM.from_pretrained benchmarking.

Quickstart (Low-VRAM Inference)

from huggingface_hub import hf_hub_download
from dr_loader import load_dr_packed

ckpt = hf_hub_download(repo_id="Grawnch/Qwen3.8-27B-DR-2bit-Packed", filename="model_packed.safetensors")
# low_vram=True keeps weights compressed in GPU VRAM (6.86 GiB); low_vram=False unpacks into standard nn.Linear
model, tokenizer = load_dr_packed(ckpt, device="cuda:0", low_vram=True)

Head-to-Head Benchmark vs. Standard 2-Bit Baselines (Qwen/Qwen3.8-27B)

Metric	16-Bit Baseline	Standard 2-Bit (RTN / Uncalibrated)	Standard 2-Bit (GPTQ)	DR (2-Bit)
Effective Bit-Width / Packed Size	16.0 bpw (51.4 GiB)	2.17 bpw (6.86 GiB)	2.17 bpw (6.86 GiB)	2.17 bpw (6.86 GiB)
Total Layer-Wise Output MSE	0.0	428,906	101,737	48,068 (2.12x)
Output Neurons Won vs. GPTQ	—	—	—	3,872,339 / 3,899,392 (99.31%)
In-Domain Perplexity (PPL)	2.518	984.4	675.2	3.587
Full-Vocab Forward KLD	0.00000	3.82910	2.84735	0.42922
Top-1 Token Agreement	100.00%	19.30%	28.95%	74.44%
Top-5 Token Agreement	100.00%	38.60%	50.00%	95.42%
Logit Cosine Similarity	1.00000	0.57941	0.67131	0.92726