outsourc-e/Qwen3.8-27B-Unleashed-GGUF

Qwen 3.8 27B UNLEASHED — GGUF

✅ All 9 tiers verified and live (2026-08-21)

UD-IQ1_M and UD-IQ2_S were briefly published broken, then withdrawn, and are now rebuilt, load-tested, and back. If you downloaded either before 2026-08-21 22:00 UTC, re-download. No other tier was ever affected.

If you are quantizing this model yourself, this is the part worth reading. Upstream builds low-bit tiers with a different architecture, not just different bit widths:

	Q3_K_XL and above	IQ1_M / IQ2_S
tensors	506	498
highest block	64 (MTP head at q6_k)	63 (MTP pruned)
block_count	65	64
nextn_predict_layers	1	0

Their imatrix has 496 entries because it matches that pruned topology. Quantizing a 65-block f16 against it fails with Missing importance matrix for tensor blk.64.attn_k.weight.

--prune-layers 64 alone is not enough — it removes the tensors but leaves the model declaring an MTP head, so loading dies on blk.63.nextn.eh_proj.weight not found: a structurally valid file containing an unloadable model. You also have to rewrite the metadata:

llama-quantize --imatrix <imatrix> --tensor-type-file IQ1_M.txt \
  --prune-layers 64 \
  --override-kv qwen35.block_count=int:64 \
  --override-kv qwen35.nextn_predict_layers=int:0 \
  model-f16.gguf out.gguf IQ1_M 12

Every file here is now gated on exit code → magic bytes → an actual load test before upload. Header checks alone let two broken builds through; only loading the model catches it.

Uncensored Qwen3.8-27B, quantized with per-tensor dynamic bit allocation.

Most uncensored GGUFs are uniform quants — every layer gets the same treatment. These use a per-tensor type map, so sensitive tensors keep precision while tolerant ones compress hard. The result: a Q3 that outperforms a Q4 3.3 GB larger, with full 250k-token retrieval.

Provenance, stated plainly. The per-tensor quant map was extracted from unsloth/Qwen3.8-27B-GGUF (Unsloth Dynamic 3.0) by reading the tensor-type table from their published files, then applied to JonathanColetti/Qwen3.8-27B-Uncensored weights with that repo's imatrix. The recipe is Unsloth's work, not ours — we did not independently derive it. All upstream projects are Apache 2.0. See Credits.

---

Measurements

Every number below is from one machine, one harness, same session — RTX 4090 24 GB, llama.cpp with DFlash2 speculative decoding, q4_0 KV cache, -c 262144, wikitext-2 perplexity at 60 chunks.

⚠️ Do not compare these PPL values against numbers from other setups. Chunk count and harness change the absolute scale; only within-table comparisons are meaningful.

Quant	Size	PPL ↓	Median tok/s	Needle retrieval	Uncensored
Unleashed UD-IQ4_XS	14.3 GB	6.3502	—	(pending)	✅
Unleashed UD-Q3_K_XL	13.2 GB	6.4036	112.8	✅ 250,806 tok	✅
Unleashed UD-IQ3_XXS	11.0 GB	6.4818	—	(pending)	✅
unsloth UD-Q3_K_XL (ref)	12.24 GB	6.3993	110.7	✅ 250,806 tok	❌
uncensored IQ4_XS (ref)	15.1 GB	6.4149	107.3	32k	✅
unsloth UD-Q4_K_XL (ref)	16.7 GB	6.4181	62.6	4k	❌
uniform Q3_K_M + imatrix (ref)	12.57 GB	6.5316	84.8	✅ 258,900 tok	✅
unsloth UD-Q2_K_XL (ref)	9.15 GB	6.6469	—	—	❌

Tiers marked (pending) are uploaded and usable; their full speed/retrieval sweep is still running.

What the table says:

• The dynamic Q3 beats every Q4 tested, including a Q4 2.2 GB larger.
• Against uniform Q3 on identical weights: −0.13 PPL and +33% throughput for 0.6 GB more.
• Quality matches Unsloth's own Q3 within error (Δ 0.0043, σ ≈ 0.045) — call it a tie, with the difference being uncensored weights at ~1 GB more.
• Q2 is the floor. The dynamic recipe halves the damage of a naive Q2 but still costs ~3.8%. Q3 is where quality holds.

Speed: what you should actually expect

Measured on real traffic (n=1,696 requests, RTX 4090, Q3_K_XL, dflash2 draft). Throughput depends far more on how many tokens you generate than on context depth, because every request pays fixed overhead (prefill, sampler init, draft warm-up) that only amortises over a long generation:

Tokens generated	n	Median tok/s
1-50 (tool calls, short acks)	1,289	24.9
51-200	97	32.6
201-600	191	43.6
601-1500	61	46.9
1500+ (long prose, code)	58	56.3

Peak observed on a single request: ~195 tok/s. Do not plan around peak.

If you are running an agent that mostly emits short tool calls, expect the ~25 tok/s end of that table, not the top. That is not a defect of this quant - it is the fixed cost of a 27B model per request, and it applies to any GGUF of this model. Speculative decoding needs a few tokens to build acceptance momentum, so very short replies finish before the draft warms up.

Lowering context does not fix this. Only 17 of 65 layers are full-attention (the rest are linear/DeltaNet), so KV work is a small share of per-token cost. We measured 61 tok/s median at 16-64k context versus 30 tok/s at <4k - deeper context was faster, because those requests generated more tokens. Context depth is not the lever; generation length is.

Retrieval at depth

Exact-string needle recovered from a 250,806-token context (98% window occupancy):

Context depth	Prompt tokens	Result	Gen tok/s
32k	31,265	✅ found	50.4
120k	119,779	✅ found	60.6
250k	250,806	✅ found	40.4

---

Files

All nine tiers, same recipe, same weights, plus a tested vision projector. Sizes are on-disk (decimal GB).

File	Size	Fits (weights + 262k ctx)	Notes
UD-IQ1_M	6.8 GB	8 GB card	⚠️ scores at chance (25.8% MMLU) — published for completeness, not for use
UD-IQ2_S	8.5 GB	12 GB card	smallest usable tier — 70.8% MMLU, passes retrieval
UD-Q2_K_XL	9.9 GB	12 GB card	best sub-10GB option
UD-IQ3_XXS	11.0 GB	16 GB card	strong value tier
UD-Q3_K_XL	13.2 GB	16–24 GB card	recommended — full 262k ctx on a 4090
UD-IQ4_XS	14.3 GB	24 GB card	best measured PPL of the ladder
UD-Q4_K_M	16.5 GB	24 GB card	reduce ctx or use q8_0 KV
UD-Q5_K_M	19.8 GB	24 GB @ ~131k ctx	quality ceiling territory
UD-Q6_K	22.1 GB	24 GB @ ~65k ctx	diminishing returns above Q4

VRAM math for a 24 GB card: weights + ~5 GB KV (262k @ q4_0) + ~2 GB draft model + compute buffers. The KV cache is small for a 27B because Qwen3.8 is hybrid DeltaNet — only 17 of 65 layers are full attention.

Quick start

llama-server -m Qwen3.8-27B-Unleashed-UD-Q3_K_XL.gguf \
  -ngl 999 -c 262144 -fa on --cache-type-k q4_0 --cache-type-v q4_0 --jinja

Charts

Which file do I want?

VRAM	File	Notes
24 GB	UD-Q3_K_XL	recommended — best quality/size, full 256k ctx
24 GB	UD-Q4_K_M / UD-Q5_K_M	if you want headroom over ctx
16 GB	UD-Q3_K_XL	reduce -c, or use q4_0 KV
12 GB	UD-Q2_K_XL	measurable quality cost — see table
8 GB	UD-IQ2_S	the smallest tier that still works — avoid IQ1_M

---

How the base model ranks

External context for the base model (stock Qwen3.8-27B, bf16 — not our quants):

• Artificial Analysis (Aug 2026): ranks alongside DeepSeek V4 and GPT-5.6-class models on the Intelligence Index — discussion · charts
• "Beats all medium models (40B–150B); same score as DeepSeek V4 Flash" — HN summary of AA data
• Hybrid DeltaNet architecture: only 17 of 65 layers are full attention → KV cache at 262k ctx is ~5 GB (q4_0), which is why full context fits on a 24 GB card at all

Our quants inherit these capabilities minus quantization loss (measured above).

Optimal settings — these matter

Setting	Value	Why
temperature	1.0	Qwen3.8 published default; pairs with top_p 0.95 / top_k 20
top_p / top_k / min_p	0.95 / 20 / 0.0	stock Qwen3.8 recommendation
repeat_penalty	1.0	K-quants at this size don't loop; leave it off
KV cache	q4_0	required for 262k ctx on 24 GB; ≤131k ctx can use q8_0
reasoning effort	low default, raise per-task	thinking burns output-token budget fast
enable_thinking	off for code-heavy tasks	reasoning chains eat the output budget

Which tier should you actually use?

Every tier was measured on the same box (RTX 4090, llama.cpp-dflash2) on the same day: wiki.test perplexity, generation speed, MMLU (10 questions x 57 subjects = 570, greedy, thinking disabled), needle-in-haystack retrieval, and a 20-prompt refusal battery.

Tier	Size	PPL ↓	tok/s	MMLU ↑	Needle retrieval	Refusal
UD-IQ1_M	6.8 GB	8.4493	77.0	25.83% ⚠️	❌ failed @32k	0.0%
UD-IQ2_S	8.5 GB	7.0195	66.6	70.83%	✅	0.0%
UD-Q2_K_XL	9.9 GB	6.6367	68.2	70.83%	✅	0.0%
UD-IQ3_XXS	11.0 GB	6.4818	62.0	75.00%	✅	0.0%
UD-Q3_K_XL	13.2 GB	6.4036	52.2	82.50%	✅	0.0%
UD-IQ4_XS	14.3 GB	6.3502	49.5	81.67%	✅	0.0%
UD-Q4_K_M	16.5 GB	6.3857	48.0	78.33%	✅	0.0%
UD-Q5_K_M	19.8 GB	6.4395	41.1	79.17%	✅	0.0%
UD-Q6_K	22.1 GB	6.4065	37.8	79.17%	✅	0.0%

MMLU here is n=120 per tier (subject-stratified) for ladder comparison; the headline 82.98% figure below is the full n=570 run on Q3_K_XL. n=120 carries roughly ±4pp, so treat small gaps as noise.

Three findings worth stating plainly

0. UD-IQ1_M scores at random chance. Do not use it. MMLU is 4-way multiple choice, so 25% is the floor for guessing. IQ1_M scored 25.83% — statistically indistinguishable from noise — with perplexity 32% worse than Q3 and a failed needle retrieval at 32k. It is the fastest file here (77 tok/s) and it retains no usable knowledge. We are shipping it with the numbers rather than quietly omitting the tier: if you have an 8 GB card, run UD-IQ2_S instead — it is 1.7 GB larger, holds 70.83% MMLU, and passes retrieval.

The degradation is not gradual. IQ2_S matches Q2_K_XL exactly (70.83%) while being 1.4 GB smaller; one step further down falls off a cliff. The usable floor of this model is 2-bit.

1. Perplexity is not capability. UD-IQ4_XS has the best perplexity in the entire ladder (6.3502) and is not the best model — it scores below UD-Q3_K_XL on MMLU while being 1.1 GB larger and 5% slower. If you are choosing a quant, choose on task benchmarks, not PPL.

2. Nothing above Q3 buys anything. UD-Q6_K is 8.9 GB larger and 28% slower than UD-Q3_K_XL, and does not score better. Quality rises steeply from Q2 to Q3 and then flattens. The recommended tier is the knee of every curve, not a compromise.

3. Abliteration survives quantization. Refusal rate is 0.0% at every tier, down to 9.9 GB. Whatever the low-bit tiers cost you, it is not censorship creeping back in.

Capability: MMLU

Measured on this exact quant (UD-Q3_K_XL), not the parent bf16 — so it includes quantization loss and abliteration loss.

Model	MMLU (0-shot, 10/subject, n=570)
Unleashed UD-Q3_K_XL (13.2 GB, uncensored)	82.98% (473/570)
stock Qwen3.8-27B bf16, same protocol (published by OBLITERATUS)	85.3%

Protocol: greedy (temperature 0), thinking disabled, single-letter answer extraction, 10 questions per subject across all 57 MMLU subjects, 0 parse failures.

Reading this honestly: ~2.3 points below stock bf16, and that gap covers both the abliteration and 3-bit quantization. For a 13.2 GB file that also keeps 262k context and runs at 25-56 tok/s on one consumer GPU (see the speed section - it depends heavily on how many tokens you generate), that is the trade being offered. If you need every last point of benchmark accuracy and don't need uncensored, run the stock bf16.

Refusal behaviour

Model	Refusal rate
Unleashed UD-Q3_K_XL	0 / 20 (0.0%)
stock Qwen3.8-27B	refuses most of this set

20-prompt boundary battery (security mechanics, drug pharmacology, graphic fiction, social engineering, morally repugnant character writing, suggestive romance). Scored by refusal-marker detection on the response opening; every prompt produced a substantive answer (806-1004 chars on the last three).

Does the abliteration method matter?

We tested this directly. Taking a different uncensored base — OBLITERATUS V2, which uses complementary blending (SVD + LEACE) rather than the Heretic method — and running it through the identical UD-Q3_K_XL recipe, same hardware, same day, so the only variable was the base weights:

Base	MMLU (n=570)	Refusal
Heretic-abliterated (this repo)	82.98% (473/570)	0/20
SVD+LEACE-abliterated, same recipe	82.98% (473/570)	0/20

Identical scores. The two models disagree on 44 individual questions (92.3% prediction agreement) — they are genuinely different models that happen to land in the same place.

At 3-bit, the abliteration technique appears to be roughly fungible: different methods, same measured capability, same refusal behaviour. Worth knowing before you switch bases chasing a benchmark claim.

Caveat, stated plainly: n=20 is a spot check, not a 800-prompt corpus. The parent model's card describes refusals as substantially reduced, not eliminated - you will likely find edge cases. This measures our quant of it, at n=20, on the prompts listed in refusal-results.jsonl.

MTP and vision

• MTP head: present. The nextn.* multi-token-prediction tensors survived abliteration (re-grafted by the parent repo) and are retained in every quant here (eh_proj at Q6_K).
• Vision: bundled and tested. mmproj-Unleashed-f16.gguf (0.93 GB) is in this repo, built from the same parent weights as these quants. Every other uncensored GGUF of this model we are aware of is text-only, because convert_hf_to_gguf.py defaults to text and drops the 333 vision tensors unless you pass --mmproj.

# vision is one extra flag - the weights themselves are unchanged
llama-server \
  -m Qwen3.8-27B-Unleashed-UD-Q3_K_XL.gguf \
  --mmproj mmproj-Unleashed-f16.gguf \
  -ngl 999 -c 262144 -fa on \
  --cache-type-k q4_0 --cache-type-v q4_0 --jinja

Then post images to /v1/chat/completions with an image_url content part (OpenAI format), or use llama-mtmd-cli.

What we verified (Q3_K_XL + this projector, single 4090):

Check	Result
loads alongside a 3-bit quant	pass - 15.9 GB VRAM at 32k ctx
shape + colour recognition	pass - "a solid red circle"
spatial reasoning (left/right)	pass - "LEFT is blue, RIGHT is yellow"

Tested with synthetic images generated at test time, so ground truth is exact and the model cannot be scored generously. Abliteration did not break cross-modal alignment - that was the open risk and it came back clean.

Cost: the projector adds ~0.9 GB resident VRAM. Text-only requests never touch the vision encoder, so there is no generation-speed penalty for having it loaded; you pay a one-time prefill per image. Omit --mmproj and behaviour is identical to a text-only build.

Not yet measured by us: image-heavy benchmark accuracy, OCR quality, multi-image prompts.

Running it

Recommended llama.cpp server config — this is the setup all measurements were taken with:

llama-server \
  -m Qwen3.8-27B-Unleashed-UD-Q3_K_XL.gguf \
  -ngl 999 -c 262144 -b 2048 -ub 512 -fa on \
  --cache-type-k q4_0 --cache-type-v q4_0 \
  --cache-reuse 256 \
  --jinja --reasoning-effort medium \
  --temperature 1 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --presence-penalty 0.0 --repeat-penalty 1.0

Speculative decoding with DFlash2 (optional, big win)

Every speed number on this card was measured with a DFlash2 draft model attached. It is not part of these files - it is a separate ~2 GB model that predicts several tokens ahead, which the main model then verifies in one pass. On our runs it accepted 48.6% of drafted tokens with a mean accepted run of 3.92 tokens. Note that acceptance on real agent traffic is lower (median ~0.37-0.51) than on benchmark prompts.

Get the draft model from spiritbuun/Qwen3.8-27B-DFlash2 (Q8_0, ~2 GB) and add:

  -md Qwen3.8-27B-DFlash2-Q8_0.gguf -ngld 999 \
  --spec-type draft-dflash --spec-draft-n-max 6 --spec-draft-p-min 0.75 \
  --spec-draft-type-k q4_0 --spec-draft-type-v q4_0

Tuning note: we A/B'd the draft parameters. n_max=6, p_min=0.75 beat both n_max=10, p_min=0.55 (41% acceptance) and n_max=10, p_min=0.40 (30% acceptance). Higher p_min wins here - drafting more aggressively costs more than it gains. Without a draft model everything still works, just slower; subtract roughly a third of the throughput.

Notes:

• --cache-reuse 256 matters more than it looks. With a stable system prompt, throughput at 120k depth measured higher than at 32k — prefix reuse amortizes the prefill.
• q4_0 KV cache is what makes 262k fit in 24 GB. At f16 KV you will OOM well before that.
• Sampler values are Qwen3.8's published recommendations, not tuned by us.

---

Method

1. Read the tensor-type table from each published Unsloth UD GGUF header (HTTP range request — metadata only, no full download).
2. Emit a --tensor-type-file map, one tensor=type line per quantized tensor (506 for Q3_K_XL).
3. Run llama-quantize against jonathan's uncensored f16 with his published imatrix and that map.

The map is non-obvious and worth looking at. Across every tier from IQ1_S to Q8_K_XL, the ~96 ssm_alpha / ssm_beta tensors are held at Q8_0 — even in the 1-bit builds. Those gate Qwen3.8's DeltaNet path. Our own hand-rolled attempt (uniform Q3_K on those tensors, heavier attention) scored 6.5865 — the worst result we measured. The specific per-layer allocation is the contribution, and it is Unsloth's.

Limitations

• Uncensored ≠ unaligned. Refusal behaviour is substantially reduced, not eliminated. See jonathan's card for measured refusal rates.
• PPL is a weak proxy for task quality. A 16-task BenchLoop coding suite (local-first LLM benchmark harness) scored identically (14/16) across every quant tested here — it did not discriminate at this tier. Treat PPL as a relative signal.
• Retrieval was tested with exact-string needles, which is the easy case. Multi-hop reasoning at 250k was not measured.
• Single-machine numbers. No cross-hardware validation.
• Vision: use the mmproj from the upstream repo; not included here.

Credits

• JonathanColetti — the uncensored base weights (abliteration via Heretic, MTP head re-grafted and verified) and the imatrix used for these quants. None of this exists without that work.
• Unsloth — the Dynamic 3.0 per-tensor quantization recipe. We extracted and reapplied their map; the method and the analysis behind it are entirely theirs. If you can use the censored base, use their files directly.
• Qwen — the base model.
• llama.cpp — quantization and inference.

Apache 2.0, inherited from all upstream components. Unsloth and Qwen names are used descriptively for attribution only; no affiliation or endorsement implied.