EliovpAI/GPT-OSS-20B-MXFP4-Paiton-RDNA4

GPT-OSS-20B on AMD RDNA4 — 2.18× speedup with Paiton

54% lower end-to-end latency on one Radeon AI PRO R9700, using OpenAI's original MXFP4 checkpoint. The stock reference already uses vLLM O2 and full decode graph capture. No new weight quantization or speculative decoding.

Primary scenario: 512 input / 256 output tokens, concurrency 1	Median latency
Fastest tested stock vLLM reference	4.819 s
Paiton	2.214 s

32 measured requests per mode after warmup; 2.176× speedup / 54.047% lower latency. The corrected-image stock run measured 6.110 s and the original-runtime repeat 6.265 s. Their unexplained timing shift is retained; the headline uses the faster 4.819-second reference. These results do not establish a universal speedup or that every possible stock setting has been exhausted.

This free community repository contains compiled artifacts and manifests, not model weights or private compiler source. The Docker image already bundles these artifacts and downloads weights directly from OpenAI at the pinned revision.

Release: v1.0.0 · GPU: R9700, gfx1201, 64 CU / 32 WGP, 32 GB VRAM · Context: 8,192 tokens including output · Scheduled requests: up to 2.

One-command Docker launch

Requires Linux x86-64, Docker and working AMD GPU access through /dev/kfd and /dev/dri. The image includes the qualified ROCm 7.14 and vLLM runtime.

docker run -d --name paiton-gptoss \
  --device /dev/kfd --device /dev/dri --group-add video --ipc=host \
  -p 8020:8020 -v paiton-gptoss-cache:/models/cache \
  ghcr.io/eliovp/paiton-vllm-plugin:gpt-oss-20b-mxfp4-rdna4-v1.0.0

The API listens on 0.0.0.0:8020. Model and compilation caches persist in the named volume. Follow docker logs -f paiton-gptoss and wait for readiness. The public checkpoint does not require a Hugging Face token. Harmony's pinned vocabulary is embedded for offline chat with cached model weights.

curl --fail http://localhost:8020/v1/models
docker exec -it paiton-gptoss python3 /opt/paiton/gpt-oss/chat.py

curl --fail http://localhost:8020/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-oss-20b","messages":[{"role":"user","content":"Explain binary search in two sentences."}],"reasoning_effort":"low","max_tokens":256,"temperature":0}'

To compare stock, stop this server and run the same Docker command with --stock after the image name. The checkpoint and serving settings remain matched. For an immutable image reference, use:

ghcr.io/eliovp/paiton-vllm-plugin@sha256:0cf9c11304ade58e91d97db876a4462781fa2c18d2d4846f26d56fe5d207669d

What Paiton optimizes

GPT-OSS has approximately 21B total / 3.6B active parameters. All experts stay on the GPU. Paiton replaces supported one/two-token expert computation with compiled RDNA4 kernels. Bulk prefill and unsupported shapes use stock experts. Attention, sinks, sliding windows, top-4 routing, sampling and Harmony parsing remain in vLLM.

Original MXFP4 storage is unpacked for FP32 accumulation, retaining the specified BF16 activation and weighted-partial boundaries. This is not native FP4 arithmetic. KV cache remains BF16. There is no requantization of the released checkpoint.

Pinned model revision: 6cee5e81ee83917806bbde320786a8fb61efebee. The upstream model supplies the weights.

Quality, memory and limits

• Stock and Paiton both pass 18/20 fixed quality tasks, with matching prompt token IDs and no task-level regressions in that set. Both retain a malformed coding-channel failure and an unconstrained JSON-fence failure.
• All 24 numerical cases and 10 API checks per mode pass, including streaming, tool continuation, reasoning efforts and concurrent long retrieval. This small suite does not prove general quality parity.
• Tested with 8,192 total context tokens, two scheduled requests and 2 GiB BF16 KV. Sampled driver memory peaks around 17 GiB; all experts are resident.
• Qualified host: 15.52 GiB RAM and 4 GiB swap. Reserve roughly 70 GB disk for the container, extracted runtime, 13.8 GB checkpoint tensors and caches.
• One cold compilation-cache startup with cached weights took about 155 s stock and 142 s Paiton. Downloads are additional; these are single observations.
• No LoRA, speculative decoding or multi-GPU qualification. Reasoning consumes the output budget; a small budget can end before a visible answer.

The primary throughput counts reasoning explicitly; it is not useful-answer throughput. Natural-stop chat and channel counts are reported separately. The ROCm kernel profiler failed, so invalid kernel traces are excluded; engine prefill/decode timings and end-to-end serving measurements are retained.

Download the compiled artifacts

The container includes the same overlay. To download it separately:

hf download EliovpAI/GPT-OSS-20B-MXFP4-Paiton-RDNA4 --revision v1.0.0 \
  --include 'overlay/*' --local-dir ./paiton-gptoss-hf

For manual use, add these options before the Docker image name:

-v "$PWD/paiton-gptoss-hf/overlay:/models/paiton-overlay:ro" \
-e PAITON_GPTOSS_ARTIFACT=/models/paiton-overlay/gptoss_mxfp4_decode_gfx1201.so

Keep the .so and .json together. The loader checks architecture, ABI and SHA-256. This is a compiled runtime overlay, not a standalone Transformers checkpoint.

Reproduce and inspect

• Model guide
• Full benchmark matrix, variability and limitations
• Reproduction and fixed evaluation set
• Machine-readable results, image pins, file checksums and artifact provenance
• Paiton community models

OpenAI's checkpoint is Apache-2.0; its license and usage policy are included. Paiton integration and compiled-runtime notices are in LICENSE, LICENSES/, NOTICE.md and THIRD_PARTY_NOTICES.md. The MIT tiktoken notice is included.