EliovpAI/Qwen3.8-27B-Quark-AWQ-MXFP4-DFlash2-Paiton-RDNA4

Qwen3.8 27B: faster serving on one R9700, inside regular vLLM

Updated 17 September 2026: published 64K and 200K images, corrected tool-call parsing, a 64K-profile quick benchmark, and an investigation of automatic prefix caching (APC) for growing conversations.

Paiton runs native HIP kernels and DFlash2 through regular vLLM with its installed library unchanged. No proprietary compiler or separate Radiance distribution is needed. This HF repository contains the original, unchanged v1.0.0 native overlay and model locks; the newer serving images are on GHCR. Target and draft weights still come from their original authors.

Choose a serving profile

Image profile	Prompt + output limit	FP8 cache	Active-sequence limit	APC default
64K v1.1.0	65,536 tokens	5 GiB	8, subject to cache capacity	Off
200K v1.1.0	200,000 tokens	8 GiB	1	Off
Original v1.0.0 benchmark image	8,192 tokens	5 GiB	8	Off

The 64K profile is the practical starting point for most users. Eight is a scheduler limit, not room for eight full-length 64K conversations. Long prompts can queue. The 200K profile serves one active request and leaves little VRAM headroom on the 32 GB R9700; keep its packaged 4,096-token prefill chunk and concurrency settings.

Both checkpoints declare 262,144 positions, but the packaged limits reflect tested serving-memory budgets. Context includes chat/tool-template tokens and generated output. A synthetic 195,999-token input plus 57-token answer passed three-marker retrieval on the 200K image; this is not broad long-context reasoning or coding qualification.

Image tags and digests · Full launch instructions · GHCR package

Run the 64K or 200K image

Use Linux x86-64, Docker, and one Radeon AI PRO R9700 with AMD GPU access. Run one server at a time. Both commands reuse the same model-cache volume. First startup downloads and verifies the pinned target and drafter, about 21.9 GB of weight files combined.

64K:

docker run --rm -d --name paiton-qwen38-agentic-64k \
  --device /dev/kfd --device /dev/dri --group-add video --shm-size 2g \
  -p 127.0.0.1:8000:8000 \
  -v paiton-qwen38-mxfp4-cache:/models/cache \
  ghcr.io/eliovp/paiton-vllm-plugin@sha256:c3ec2528285b484b2e0af7f571f80f1da23c1970210e4d3d09f5d2a909414186

200K, one active request:

docker run --rm -d --name paiton-qwen38-agentic-200k \
  --device /dev/kfd --device /dev/dri --group-add video --shm-size 2g \
  -p 127.0.0.1:8000:8000 \
  -v paiton-qwen38-mxfp4-cache:/models/cache \
  ghcr.io/eliovp/paiton-vllm-plugin@sha256:28af1731cfd8aceab51915411e8f879531128c2711b4193c76c5b6eba8ba2ef4

Wait for startup and curl --fail http://127.0.0.1:8000/health. Use base URL http://127.0.0.1:8000/v1 and model Qwen3.8-27B-Quark-AWQ-MXFP4. Stop the selected container before switching profiles; the cache volume persists. No model weights or native-library bytes changed for these profile releases.

Tool calling and thinking

The new images select qwen3_xml tool parsing and qwen3 reasoning parsing. The original parser mismatched the checkpoint's XML tool format, leaving tool syntax as text instead of structured API calls. The correction passed the bounded streaming/non-streaming tool tests and an actual OpenCode 1.18.31 file read/write check. This fixes server-side parsing, not model weights.

Thinking is disabled by default in both v1.1.0 profiles. A request can enable it with "chat_template_kwargs": {"enable_thinking": true}. Tool-test evidence and limits.

Quick benchmark on the 64K image

Concurrent requests	Median generation per request, tok/s	Aggregate output, tok/s	Median client TTFT, ms
1	99.95	84.42	106.29
2	89.55	152.69	168.05
4	86.16	230.76	182.87
8	67.81	304.10	350.84

BetterBench 0.6.0: 52 measured requests, 10 excluded warmups, 128-token output cap, thinking disabled; zero request errors or preemptions. Concurrency prompts were only 69–116 tokens. These are short-prompt results on a 64K-capable image, not 64K-prompt throughput. 37/52 outputs reached the cap, so this does not establish completed-task quality. The changed thinking default also prevents treating this as a matched-settings speedup over the old image.

Full report, raw measurements and reproduction · SVG.

Prefix caching: much shorter waits when history repeats

APC reuses an unchanged cached prefix instead of reprocessing the whole conversation. It can substantially reduce waiting in long coding sessions. Our default compact native GDN path currently disables APC because it does not yet support this prefix-reuse state contract.

Configuration	Cold → repeated-prefix TTFT	Availability
Stock GDN + APC, approximately 40K prompt, 5 GiB cache	15.58 s → 1.13 s	Published 64K image with explicit profile override
Native-prefill APC candidate, approximately 150K prompt, 8 GiB cache	88.46 s → 2.04 s; independent repeat 88.61 s → 2.03 s	Experimental adapter, not distributed

The approximately 150K tests used a 160K context limit and one active request. The 43–44× improvement is cold-versus-cache-hit response-start latency for that candidate; it is not a decode-throughput gain or an out-of-box image result.

The published images still default to APC off and do not implement PAITON_PREFIX_CACHING=1. Their container launchers do not accept --enable-prefix-caching directly either. Use the published stock-GDN APC profile-override recipe, which enables vLLM prefix caching and aligned state caching while disabling the incompatible native GDN registrations. Other Paiton optimizations and DFlash2 remain active. No unpublished adapter is required for this stock-GDN recipe.

APC is a workload option: the current fallback gives up decode speed and usable cache capacity to accelerate repeated histories. At the same 5 GiB budget, reported capacity was 119,088 token-equivalents for compact/APC-off versus 81,727 for stock-state/APC-on. Fresh prompts, eviction or changed early tokens still need cold processing. The report retains the decode probes and their sampling/acceptance limits.

Do not apply this fallback to the 200K profile with its existing 8 GiB cache. The pinned stock-APC sizing calculation needs about 9.163 GiB before its null-block margin; a larger allocation has not been qualified on this card.

APC report, tradeoffs and raw evidence · Reproduce stock APC and controls · SVG.

Original 8K release: historical comparisons

The comparisons below used the original 8,192-token context limit and 5 GiB cache, with their original sampling and workload settings. They are separate from the v1.1.0 quick benchmark and APC experiments above.

Paiton delivers 57% more throughput at eight concurrent requests, 22% higher weighted decode throughput, and 12.5–17.3% faster prefill than Radiance + DFlash2 on the same Radeon AI PRO R9700. These gains are measured with the same Qwen3.8-27B MXFP4 checkpoint and cache budget across a full 188-request benchmark.

The cache improvement changes the experience under load: Paiton provides 2.89× the estimated token capacity in the same 5 GiB cache pool, and median time to first token at eight concurrent requests falls from 6.59 seconds to 195 ms, including queueing. The engine can keep more requests active instead of making them wait.

This package combines Paiton's native HIP kernels, techniques adapted from Radiance and libr4d, and DFlash2 through the official vLLM runtime. The installed vLLM library stays unchanged. There is no separate Radiance engine or DFlash package to install.

This repository contains a small native runtime overlay, locked upstream model references, checksums, and benchmark evidence. Target and draft weights download from their original authors; this repository does not duplicate them or introduce new fine-tuning. No compiler checkout or compilation step is needed to serve it.

The original v1.0.0 image remains available. The image is pinned by digest below; its release audit verifies unchanged benchmarked native binaries and the packaged API launch.

Three engines, the same checkpoint

The common 54-request matrix compares stock vLLM O2, the recommended Radiance profile with DFlash2, and Paiton on regular vLLM with DFlash2. At eight concurrent requests, throughput is 33.7 / 175.6 / 328.5 tok/s respectively. This matrix uses a 128-token generation cap. Stock's checkpoint-native W4A4 emulation and the accelerated W4A8 paths are different complete engine configurations on the same model weights.

Full-workload results against Radiance

Both accelerated engines complete the longer 188-request preset. Paiton leads all eight task categories, all four concurrency throughput levels, and all three prefill depths in this comparison.

Full-workload metric	Radiance + DFlash2	Paiton + DFlash2
Weighted serial decode	86.0 tok/s	104.9 tok/s
One concurrent request	76.9 tok/s	89.6 tok/s
Two concurrent requests	142.7 tok/s	165.7 tok/s
Four concurrent requests	187.1 tok/s	254.8 tok/s
Eight concurrent requests	200.3 tok/s	314.5 tok/s
Prefill, approximately 1,556 input tokens	2,828.4 tok/s	3,181.5 tok/s
Prefill, approximately 3,024 input tokens	3,002.7 tok/s	3,521.2 tok/s
Prefill, approximately 5,226 input tokens	2,925.8 tok/s	3,367.2 tok/s
Estimated token slots in the 5 GiB cache pool	25,746	74,430

Time to first token includes queueing; lower is better.

Concurrent requests	Radiance median TTFT	Paiton median TTFT
One	94.8 ms	105.0 ms
Two	148.6 ms	159.9 ms
Four	1,626.8 ms	180.1 ms
Eight	6,585.5 ms	194.8 ms

Radiance retains an approximately 10–11 ms TTFT advantage at one and two concurrent requests. Paiton's much larger latency gains appear under concurrent load. The cache figure is a capacity estimate; the tested per-request context limit remains 8,192 tokens.

Complete benchmark tables and methodology · Machine-readable metrics · CSV.

Run the original 8K benchmark image

Use Linux x86-64, Docker, and one Radeon AI PRO R9700 with working AMD GPU device access. The dedicated image includes the qualified official vLLM 0.28 ROCm runtime and Paiton plugin. The first start downloads and verifies the pinned target and DFlash2 snapshots, approximately 21.9 GB of weight files combined.

docker run -d --name paiton-qwen38-mxfp4 \
  --device /dev/kfd --device /dev/dri --group-add video --shm-size 2g \
  -p 127.0.0.1:8000:8000 \
  -v paiton-qwen38-mxfp4-cache:/models/cache \
  ghcr.io/eliovp/paiton-vllm-plugin@sha256:9b2dae214076d35de785e073b31294b033a376b16e6bc1ec1fdada4e54d96c59

Follow docker logs -f paiton-qwen38-mxfp4 until startup completes, then check curl --fail http://127.0.0.1:8000/health. Later starts reuse the cache.

curl --fail http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Qwen3.8-27B-Quark-AWQ-MXFP4","messages":[{"role":"user","content":"Write a Python function that removes duplicates while preserving order."}],"temperature":0,"max_tokens":256,"stream":true,"chat_template_kwargs":{"enable_thinking":false}}'

This original v1.0.0 launch uses eight concurrent requests, an 8,192-token context limit, FP8 KV storage, a 5 GiB cache pool, up to 4,096 tokens per prefill chunk, and DFlash2 with seven speculative tokens and unpadded drafting. Prefix caching and async scheduling are disabled in the tested profile. Text generation is qualified here; this package does not claim image-input support or support for other GPU architectures.

The image invokes the ordinary vllm.entrypoints.openai.api_server entry point. The deployment check covers streaming, eight concurrent requests, the 8K context boundary, and a successful fresh request afterward. Deployment check · Unchanged-vLLM audit.

Download the native overlay

All three images include these unchanged native artifacts. The download below intentionally stays pinned to the original v1.0.0 overlay: this documentation refresh does not publish a new overlay or APC runtime. To obtain the exact payload and verify its file checksums separately:

hf download EliovpAI/Qwen3.8-27B-Quark-AWQ-MXFP4-DFlash2-Paiton-RDNA4 \
  --revision v1.0.0 --local-dir ./paiton-qwen38-hf
cd paiton-qwen38-hf
sha256sum -c SHA256SUMS

Keep overlay/ intact. For an explicit overlay deployment, mount that directory read-only at /opt/paiton-native in the same qualified image. The image's bundled payload is the recommended default. These native libraries load through the Paiton plugin; this repository is not a standalone Transformers checkpoint.

Paiton's native libraries build and load independently of PyTorch, Triton, and Radiance. The external serving adapter retains vLLM's standard framework dependencies and DFlash2 scheduling. The proprietary compiler stays private.

Model authors and provenance

• Target checkpoint and quantization: amd/Qwen3.8-27B-Quark-AWQ-MXFP4.
• Draft checkpoint: tcclaviger/Qwen3.8-27B-DFlash2-FP8.
• Runtime integration: Eliovp-BV/paiton-vllm-plugin, built on vLLM.
• Adapted techniques: magiccodingman/vllm-radiance and StillDeadcode/libr4d.

See checkpoint hashes, runtime manifest, release provenance, and third-party attribution and terms.

These are same-GPU measurements of the configurations documented here. Public two-GPU Radiance numbers and measurements of other checkpoints are separate comparisons; no two-GPU scaling extrapolation or worldwide performance ranking is claimed.

Explore Paiton · Optimized-model library