TL;DR

I benchmarked Qwen3.8-27B on a single RTX 3090 24GB using llama.cpp, comparing normal Q4 decoding, built-in MTP speculative decoding, and DFlash2.

My best practical configuration so far is:

• Target: Qwen3.8-27B-UD-Q4_K_XL
• DFlash2 drafter: Qwen3.8-27B-DFlash2-Q8_0
• Context: 163,840 (~160K)
• KV cache: Q4_0
• DFlash n_max: 4
• VRAM: ~23.08 GB
• Sustained generation: 59.88 tok/s
• GPU: RTX 3090 24GB @ 300W

Compared with:

So on my setup, Q8 DFlash2 increased sustained generation throughput by ~81% over normal Q4 decoding and ~24% over MTP2, while retaining a 160K context window on a single 24GB 3090.

A few important findings:

• Q8 DFlash2 works fine with a Q4 target — the draft quant does not need to match the target.
• Q8 was actually faster than Q4 as the DFlash2 drafter on my 3090 because its higher draft accuracy/acceptance outweighed the extra compute.
• n_max=4-5 was the sweet spot. Increasing to 7 produced longer accepted sequences but reduced overall throughput.
• Increasing context from 160K to ~176K worked, but VRAM increased to ~23.46 GB and throughput fell to ~57 t/s.
• Very short benchmarks were misleading. The useful results came from ~10K-token coding generations.
• DFlash2 required the appropriate llama.cpp support; an older build failed with wrong number of tensors; expected 81, got 58.

Current daily driver: Q4_K_XL target + Q8_0 DFlash2 + 160K context + Q4 KV + n_max=4 = ~60 tok/s on one RTX 3090.

Full testing, configuration, VRAM measurements, acceptance rates and n_max results below.

Qwen3.8-27B + DFlash2 on RTX 3090 24GB — llama.cpp real-world benchmarks (~60 tok/s @ 160K)

I've been testing Qwen3.8-27B with DFlash2 on a single RTX 3090 24GB using llama.cpp.

The goal was to find a practical configuration for agentic coding / repository work: good model quality, large context, and high generation speed, without needing multiple GPUs.

After testing normal Q4 decoding, MTP2, DFlash2 Q4, DFlash2 Q8, different context sizes, and several DFlash n_max values, my current best practical result is:

Qwen3.8-27B UD-Q4_K_XL + DFlash2 Q8_0 + 160K context = ~59.9 tok/s sustained generation on a single RTX 3090.

This post documents the setup, results, problems encountered, and tuning.

---

Hardware

GPU:

NVIDIA GeForce RTX 3090 24GB

GPU power limit:

300W

CPU:

Intel Xeon Gold 6248

CPU threads given to llama.cpp:

20

NVIDIA driver:

595.84

Host nvidia-smi reports:

CUDA Version: 13.2

The GPU is dedicated to llama.cpp during these tests.

---

Models

Target model:

Qwen3.8-27B-UD-Q4_K_XL.gguf

DFlash2 models tested:

Qwen3.8-27B-DFlash2-Q4_K_M.gguf
Qwen3.8-27B-DFlash2-Q8_0.gguf

I also compared DFlash2 against normal decoding and the model's built-in MTP speculative decoding.

One important finding is that the DFlash2 quantization does NOT need to match the target model.

My best configuration uses:

Target: Q4_K_XL
Draft:  Q8_0

The Q8 model is only the speculative drafter. The final output is still produced/verified by the Q4 target model.

---

llama.cpp configuration

Common settings were:

--n-gpu-layers all
--threads 20
--cache-type-k q4_0
--cache-type-v q4_0
--flash-attn on
--parallel 1
--reasoning off
--jinja

For DFlash2:

--model-draft /models/Qwen3.8-27B-DFlash2-Q8_0.gguf
--n-gpu-layers-draft all
--spec-type draft-dflash
--spec-draft-n-max 4

My current 160K configuration is essentially:

-m /models/Qwen3.8-27B-UD-Q4_K_XL.gguf
--model-draft /models/Qwen3.8-27B-DFlash2-Q8_0.gguf
--n-gpu-layers all
--n-gpu-layers-draft all
--threads 20
--cache-type-k q4_0
--cache-type-v q4_0
--flash-attn on
--ctx-size 163840
--parallel 1
--reasoning off
--spec-type draft-dflash
--spec-draft-n-max 4
--jinja

Sampling settings were kept constant:

temperature: 0.7
top-p: 0.8
top-k: 20
min-p: 0.0
repeat penalty: 1.0
presence penalty: 1.5

---

Test methodology

I found that short generations can give a misleading picture of speculative decoding performance.

For the useful comparisons I used a substantial coding prompt asking Qwen to produce a production-style FastAPI application including things such as:

• Python 3.12
• FastAPI
• SQLAlchemy 2 async
• PostgreSQL
• Pydantic v2
• CRUD
• repositories
• services
• pagination/filtering
• structured logging
• pytest
• integration tests
• Dockerfile
• docker-compose
• configuration

The important part was generating approximately 10K+ output tokens.

This lets the reported final eval time / tokens-per-second settle rather than relying on short bursts.

I tested:

1. Q4 normal decoding
2. Q4 + MTP2
3. Q4 + DFlash2 Q4
4. Q4 + DFlash2 Q8
5. 128K / 160K / ~176K context
6. Different DFlash2 n_max settings

All tests used:

--parallel 1

---

Main results

So the best configuration I've tested so far is:

Q4_K_XL target
+
Q8_0 DFlash2
+
163840 context
=
59.88 t/s

---

Performance improvement

Using my normal Q4 result as the baseline:

Normal Q4
33.16 t/s
1.00x

Q4 + MTP2
48.32 t/s
~1.46x

Q4 + DFlash2 Q8 @ 160K
59.88 t/s
~1.81x

So the best DFlash2 configuration was approximately:

81% faster than normal Q4 decoding

and approximately:

24% faster than my MTP2 result.

This was a much larger improvement than I initially expected from a single RTX 3090.

---

Q4 vs Q8 DFlash2 drafter

This was one of the most interesting tests.

Initially I used:

Qwen3.8-27B-DFlash2-Q4_K_M.gguf

At 160K I measured approximately:

Generation:       56.06 t/s
Draft acceptance: 72.2%
Mean draft length: 3.89
VRAM:             ~22.25 GB

I still had unused VRAM, so I changed ONLY the DFlash2 model to:

Qwen3.8-27B-DFlash2-Q8_0.gguf

The target remained:

Qwen3.8-27B-UD-Q4_K_XL.gguf

At the same 160K context:

Generation:       59.88 t/s
Draft acceptance: 77.2%
Mean draft length: 4.09
VRAM:             ~23.08 GB

So:

DFlash Q4: 56.06 t/s
DFlash Q8: 59.88 t/s

That's approximately a 6.8% throughput improvement from using the higher-quality Q8 drafter.

The Q8 draft consumed approximately another 0.8GB of VRAM, but on my 24GB 3090 that was a good trade.

It appears the improved draft accuracy/acceptance was enough to outweigh the additional cost of running the larger draft quant.

Again, this does NOT make the final model "Q8 quality".

The target model is still:

Qwen3.8-27B-UD-Q4_K_XL

The Q8 model is only predicting speculative candidates.

---

DFlash2 n_max testing

I also tested different maximum speculative draft lengths.

This produced an interesting result:

Longer speculative sequences do not automatically mean higher throughput.

At approximately 176K context using the Q8 drafter:

n_max=4 and n_max=5 are effectively tied at this context.

The ~0.46% difference between:

56.96 t/s

and:

57.22 t/s

is small enough that I would not claim n_max=5 is definitively faster without repeated runs.

However, n_max=7 was clearly worse.

---

Why longer drafts weren't necessarily faster

Increasing n_max increased the mean speculative sequence length:

n_max 4:
mean length = 3.90

n_max 5:
mean length = 4.42

n_max 7:
mean length = 5.51

But draft acceptance simultaneously declined:

n_max 4:
~72.4%

n_max 5:
68.38%

n_max 7:
64.45%

The n_max=7 long generation reported:

9817 accepted / 15232 generated
acceptance = 0.64450
mean len = 5.51

and:

11993 generated target tokens
54.94 tokens/sec

So DFlash2 was successfully creating longer accepted sequences, but it was doing substantially more speculative work to obtain them.

That additional work isn't free on the RTX 3090.

The result is an important distinction:

longer accepted speculative sequences != higher end-to-end throughput.

For my hardware, the sweet spot appears to be around n_max=4-5.

---

n_max=8 / OOM result

I also tried:

--spec-draft-n-max 8

DFlash2 reports a trained block size of 8:

block_size=8

llama.cpp therefore reported:

requested draft size (n_max=8, n_min=0)
exceeds the trained block size 8 -- clamping to 7

At approximately 176K context with the Q8 drafter, this run subsequently hit:

CUDA error: out of memory

and exited.

So there is no reason on this configuration to attempt n_max=8.

n_max=7 is already slower than 4/5 and pushes the GPU harder.

---

Detailed n_max=5 result

The successful n_max=5 long generation produced:

prompt tokens: 889
generated tokens: 10111

prompt eval:
456.53 t/s

generation:
57.22 t/s

total:
11000 tokens

draft acceptance:
7823 accepted / 11440 generated

acceptance:
68.383%

mean draft length:
4.42

This was at:

n_ctx_slot = 180224

with:

Qwen3.8-27B-UD-Q4_K_XL
+
Qwen3.8-27B-DFlash2-Q8_0

---

Detailed n_max=7 result

The n_max=7 run produced:

prompt tokens: 889
generated tokens: 11993

prompt eval:
465.43 t/s

generation:
54.94 t/s

total:
12882 tokens

draft acceptance:
9817 accepted / 15232 generated

acceptance:
64.450%

mean draft length:
5.51

Again this was:

n_ctx_slot = 180224

The instantaneous tg_3s number occasionally exceeded 70 t/s during the run, but the important number for comparison is the final sustained generation rate:

54.94 t/s

This is why I think long generations are important when benchmarking speculative decoding.

---

Context scaling

I also tested how far I could push context while keeping both the Q4 target and Q8 DFlash2 model on the RTX 3090.

My measured results were approximately:

160K:
VRAM = 23.08 GB
generation = 59.88 t/s

~176K:
VRAM = 23.46 GB
generation = ~57 t/s

The 176K configuration is:

--ctx-size 180224

and llama.cpp confirms:

n_ctx_slot = 180224

So 176K definitely works on this card.

However, I don't think maximizing context is the best use of the remaining VRAM.

Going from 160K to 176K:

VRAM:
23.08 -> 23.46 GB

while throughput dropped approximately:

59.88 -> 56.96 t/s

That's roughly a 5% performance reduction for another 16K context.

It also leaves very little free VRAM for temporary CUDA allocations.

---

160K appears to be the practical sweet spot

For my coding workload I therefore prefer:

163840 context
~23.08 GB VRAM
~59.9 t/s

rather than:

180224 context
~23.46 GB VRAM
~57 t/s

The extra 16K context isn't particularly valuable to me compared with:

• higher generation speed
• additional CUDA headroom
• lower chance of OOM
• more predictable operation

176K is useful as a long-context profile, but I wouldn't use it as my normal configuration.

---

One llama.cpp message that initially confused me

During startup I see:

dflash requires ctx_other to be set
(this warning is normal during memory fitting)

Initially I thought this meant DFlash2 wasn't working.

It doesn't.

After the memory-fitting stage the server continues and reports things such as:

loading draft model '/models/Qwen3.8-27B-DFlash2-Q8_0.gguf'

initializing, n_slots = 1, n_ctx_slot = 180224

adding speculative implementation 'draft-dflash'

n_max=5
block_size=8
mask_token_id=248070
n_extract=5
sample_from_anchor=true

and then:

model loaded

So the earlier ctx_other message during memory fitting is not itself evidence that DFlash2 failed.

---

Another llama.cpp issue I encountered

Before getting the correct DFlash2-capable llama.cpp build running, I encountered:

wrong number of tensors; expected 81, got 58

when attempting to load the DFlash2 GGUF.

I needed a llama.cpp build containing the DFlash2 support I was testing rather than simply assuming my existing llama.cpp Docker image supported the model format.

Once the correct build was running, the draft model loaded correctly and llama.cpp explicitly reported:

adding speculative implementation 'draft-dflash'

So if someone sees the tensor-count error, check the exact llama.cpp version/branch before assuming the GGUF itself is corrupt.

---

Current daily-driver configuration

Based on all the testing so far, this is the configuration I'm currently keeping for coding:

GPU:
RTX 3090 24GB

Target:
Qwen3.8-27B-UD-Q4_K_XL.gguf

Drafter:
Qwen3.8-27B-DFlash2-Q8_0.gguf

Context:
163840

KV:
q4_0 / q4_0

DFlash:
n_max = 4

Flash Attention:
ON

GPU layers:
all

Draft GPU layers:
all

Parallel:
1

Reasoning:
OFF

VRAM:
~23.08 GB

Sustained long-generation performance:
~59.9 t/s

Why n_max=4 rather than 5?

At 176K, 4 and 5 were effectively tied:

n_max 4 = 56.96 t/s
n_max 5 = 57.22 t/s

But my best actual 160K measurement was obtained using n_max=4:

59.88 t/s
77.2% acceptance
mean length 4.09

So I'm keeping n_max=4 until I have a direct 160K n_max=5 A/B test showing otherwise.

---

Summary of what I learned

For my RTX 3090 24GB:

1. DFlash2 works extremely well with Qwen3.8-27B.

Normal Q4 generation was around:

33.16 t/s

My best DFlash2 result was:

59.88 t/s

That's approximately an 81% improvement.

2. DFlash2 substantially outperformed MTP2 in my test.

MTP2:    48.32 t/s
DFlash2: 59.88 t/s

Approximately 24% higher generation throughput.

3. A Q8 DFlash2 drafter works with a Q4 target.

And on my 3090 it actually performed better than the Q4 DFlash2 drafter.

At 160K:

DFlash Q4:
56.06 t/s
72.2% acceptance

DFlash Q8:
59.88 t/s
77.2% acceptance

4. Spare VRAM can be better spent on draft quality than additional context.

Moving the DFlash2 drafter from Q4 to Q8 consumed roughly another 0.8GB but improved throughput.

By contrast, pushing context from 160K to 176K consumed additional VRAM while reducing throughput.

5. Maximum draft length is not necessarily optimal.

At ~176K:

n_max 4: 56.96 t/s
n_max 5: 57.22 t/s
n_max 7: 54.94 t/s

The practical sweet spot on my card appears to be around 4-5.

6. Acceptance percentage alone isn't enough.

You need to look at:

• sustained generation t/s
• draft acceptance
• mean accepted length
• number of speculative tokens generated
• VRAM usage

The highest mean draft length did not give me the highest throughput.

7. Long benchmarks matter.

Short responses gave highly variable results.

10K+ token coding generations gave a much better picture of actual sustained throughput.

---

Final result

For my workload — mainly agentic coding and repository work — the configuration to beat is currently:

Qwen3.8-27B UD-Q4_K_XL
+
Qwen3.8-27B DFlash2 Q8_0
+
160K context
+
Q4 KV
+
n_max=4
+
single RTX 3090 24GB

Result:

~23.08 GB VRAM
~59.9 tok/s sustained generation

Compared with approximately:

~33.2 tok/s without speculative decoding.

So for this workload DFlash2 is giving me roughly 1.8x the normal Q4 generation throughput while retaining 160K context on a single 24GB Ampere card.

Hopefully these numbers are useful to anyone else experimenting with Qwen3.8-27B / DFlash2 / llama.cpp on a 3090.

I'm interested to see results from other 3090/4090 users, particularly different DFlash2 quants, n_max values, and context sizes.