Qwen3.8-27B-ASCII-Condensed

Qwen3.8-27B with a condensed ASCII-only vocabulary and a matching DFlash2 draft model. Meant to be used together with troed/llama.cpp-adaptive-kv-streaming, a llama.cpp fork that adds Raymond's KV streaming plus my own speculative draft support.

• vocabulary reduced from 248,320 to 129,006 rows in the embedding and output head
• the freed VRAM becomes KV cache: 160K context on a 16 GB GPU
• speculative decoding with the included DFlash2 draft (spec-type = draft-dflash)
• no retraining: surviving weights are bit-identical to the sources below

Files

File	Size	Role
Qwen3.8-27B-ASCII-Condensed-IQ4_XS-3.84bpw.gguf	11.4 GiB	target model
Qwen3.8-27B-ASCII-Condensed-DFlash2-Q2_K_S-MIX.gguf	511 MiB	DFlash2 draft model (load with md =)

How the files were made

Target

Created from byteshape/Qwen3.8-27B-GGUF (its Qwen3.8-27B-IQ4_XS-3.84bpw.gguf) using bsaleh03's ASCII-Condensed-prune-tools. The vocab rows were gathered directly in quantized space and the tokenizer was rewritten to match, so every surviving weight is bit-identical to the source: no dequantization, no requantization, no retraining.

Draft

Created (scripts in my fork repo) from the original HermiHg/Qwen3.8-27B-DFlash2-Q2_K_S-MIX-GGUF, with the same condensation applied to the draft's vocab tensors and tokenizer. No weights were changed beyond the row subset.

Requirements

A build of troed/llama.cpp-adaptive-kv-streaming:

cmake -B build -DGGML_NATIVE=ON -DLLAMA_BUILD_EXAMPLES=OFF -DLLAMA_BUILD_TESTS=OFF -DGGML_CUDA_FA_ALL_QUANTS=ON -DGGML_CUDA=ON
cmake --build build --config Release -j

The files are standard GGUFs and load in any recent llama.cpp, but the kv-stream-* options and the pinned-draft behavior need the fork.

Usage

hf download troed/Qwen3.8-27B-ASCII-Condensed --local-dir models

Config for the fork's llama-server (local paths adjusted):

[Qwen3.8-27B]
chat-template-file = chat_template_qwen3.8.jinja
m = models/Qwen3.8-27B-ASCII-Condensed-IQ4_XS-3.84bpw.gguf
md = models/Qwen3.8-27B-ASCII-Condensed-DFlash2-Q2_K_S-MIX.gguf
device-draft = CUDA0
n-gpu-layers-draft = all
spec-type = draft-dflash
spec-draft-n-max = 5
ctx-size = 160000
n-gpu-layers = 99
batch-size = 256
ubatch-size = 256
cache-type-k = q8_0
cache-type-v = q4_0
kv-stream-arena-mib = 4352
kv-stream-spec-dynamic = on
kv-stream-spec-keep-pages = 334
kv-stream-spec-reenable-pages = 8
kv-stream-spec-stable-decodes = 4
fit = off
parallel = 1
temp = 1.0
top-p = 0.95
top-k = 20
min-p = 0.0
presence-penalty = 0.0
repeat-penalty = 1.0
reasoning = on
reasoning-preserve = on
no-mmproj-offload = on
mmproj = Qwen3.8-mmproj-BF16.gguf
load-mode = none
flash-attn = on

chat-template-file is any Qwen 3.8 jinja chat template. No mmproj is provided here; use e.g. the one from byteshape/Qwen3.8-27B-GGUF (mmproj-bf16.gguf), or drop the mmproj/no-mmproj-offload lines for text only.

Performance

On an RTX 5060 Ti 16 GB + 96 GB DDR5, with the config above:

• prompt processing: ~600-900 t/s
• token generation: ~15-50 t/s

Full setup walkthrough: 16 GB VRAM llama-server configs.

Language support

The vocabulary is ASCII only. The 256 byte-level fallback tokens are always kept, so non-ASCII text still decodes correctly, it just costs more tokens per character. If you need Latin-extended, Greek, currency or box-drawing characters, re-run the prune tools with a wider policy.

Credits

• Qwen for Qwen3.8-27B
• Raymond Huang for the KV cache streaming work
• ByteShape for the IQ4_XS-3.84bpw quantization (weights unaltered)
• bsaleh03 for the vocabulary pruning tools
• HermiHg for the Q2_K_S-MIX DFlash2 draft (weights unaltered)

Licensed under Apache-2.0, inherited from the base model.