Qwen3.8-Flash-Next (Qwen 4 - Preview) vs 3.8 27B - Architecture Teardown
I could not run this (360 GB against a 32 GB card), so I read config.json andmodel.safetensors.index.json for Flash Next and for Qwen 3.8 27B and compared them.
Everything below is counted from those two files, not read off the model card.
Most of the model is untouched: same 248,320 vocab and bos/eos id, same 3 linear to 1
full attention pattern, head_dim 256, 24 query heads, rope_theta 1e7, 262,144
context, same vision tower. The stack is smaller, 48 blocks at 2,560 wide against 64
at 5,120. Four parts changed.
| Part | Qwen 3.8 27B | Qwen3.8-Flash-Next |
|---|---|---|
| Feed forward | dense MLP, 17,408 wide, all active | 512 experts of 640, 10 routed + 1 shared. 121B stored, 2.6B run |
| Attention | 16 full blocks, whole cache, KV 64 KiB/tok | 12 blocks behind a DSA indexer (indexer_budget 2048), num_key_value_heads 2, KV 24.75 KiB/tok |
| Normalization | input_layernorm + post_attention_layernorm |
hyper connections, hc_count 4, hc_lowrank 320, four residual lanes at 10,240 |
| Extra capacity | none | PLE at layer 2, ngram_size 3, 128 shards, 51.2B params with no matmul |
Per decoded token at 262k: KV read is 16.0 GiB against roughly 240 MiB, and total
traffic at bf16 is 66.4 GiB against 11.5 GiB. But this is not constant cost attention.
The indexer still scans at 768 bytes per token, so the curve rises, just 85 times more
slowly.
Two questions:
What separates DeepSWE from SWE-bench Pro? DeepSWE gains 16.5 points and
SWE-bench Pro gains 0.8, and the card says both ran at 256k. If depth were the whole
mechanism, those should move together.
Is anyone working on host side PLE offload? The 51B table is a third of the
storage and a pure lookup, so it is the one part that suits system RAM. That looks
like the difference between needing 100 GB of VRAM and a machine people already own.
Full teardown: https://kgptalkie.com/tutorials/generative-ai/qwen-3-8-flash-next-vs-qwen-3-8-27b
how about offloading the 51B to a fast SSD?
how about offloading the 51B to a fast SSD?
Yes that is possible. Upcoming Qwen 4 seems quite promising.
damn I wish I could run this on my 24gb vram gpu 😢 like qwen3.8-27b
I tested Qwen3.8-Flash-Next (177B) on a single desktop:
• 4-bit: Impossible on 96 GB RAM
• 2-bit CPU: 11.1 t/s
• 1-bit CPU: 10.8 t/s
• 2-bit + GPU offload: 43.7 t/s
• 1-bit + GPU offload: 52.0 t/s
That's up to 4.8× faster with GPU offload.
The takeaway: quantization + the right GPU offload point matters more than just throwing more VRAM at it.
177B on a desktop is getting seriously interesting.
Full benchmarking is coming soon.
Keep an eye at https://kgptalkie.com/tutorials/llm-benchmarking
I tested Qwen3.8-Flash-Next (177B) on a single desktop:
• 4-bit: Impossible on 96 GB RAM
• 2-bit CPU: 11.1 t/s
• 1-bit CPU: 10.8 t/s
• 2-bit + GPU offload: 43.7 t/s
• 1-bit + GPU offload: 52.0 t/s
That's up to 4.8× faster with GPU offload.The takeaway: quantization + the right GPU offload point matters more than just throwing more VRAM at it.
177B on a desktop is getting seriously interesting.Full benchmarking is coming soon.
Keep an eye at https://kgptalkie.com/tutorials/llm-benchmarking
how about the intelligence loss at 2bit ? anything below 4bit is commonly not advised for real work (agentic), maybe we can keep the experts at vram and the rest offload to cpu? idk
(i get like 45-50 t/s with qwen 3.8-27b 4bit, full gpu vram btw)
A guy on YouTube was saying raising the indexer_budget solved his hallucination problem in SGlang. Is that right? Can it be done with llama.cpp? The model is terrible after 150k context (Q4_K_XL)
Edit - I gave more details here https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF/discussions/42#6a9229450a91fbd28c0db571 (in case this question doesn't belong here)